4 Recognition Modules - Tabjones/pacman_vision GitHub Wiki

Recognition Modules provide two additional modules for object recognition, pose estimation and 3D object tracking from the processed cloud stream. These are the Estimator Module and the Tracker Module.

To enable both you have to set the CMake variable

    PACV_BUILD_RECOGNITION

to 1 from the top-level CMakeLists.txt or by passing -DPACV_BUILD_RECOGNITION=1 to CMake command line.

These two Modules require an additional dependency: Pose Estimation Library (PEL), to install it follow the next Section.

Pose Estimation Library (PEL)

PaCMaN Vision uses PEL to perform online pose estimation of objects and tracking, this works via a database of known object poses, which PaCMaN Vision build process already downloads for you. The database contains some Ikea kitchen objects like mugs and containers, however if you want to use or create a different database you are encouraged to read PEL wiki pages.

To install PEL just clone it into your catkin workspace and let it build along side your workspace with catkin build:

roscd && cd ../src
git clone https://github.com/Tabjones/Pose-Estimation-Library pel
cd ..
catkin build

Note To be able to build PEL into the catkin workspace you have to use catkin build command from catkin_tools package, catkin_make won't be able to handle a pure CMake package like PEL.

If you don't want to use catkin_tools you must install PEL into the system, for example into /usr/local prefix. Instructions on how to do it are found on PEL project page.

Asus Scanner Models

You will also need this package at runtime, because it provides the meshes of the objects you are identifying/tracking, and PaCMaN Vision expects to find those meshes.

Just clone the package into your catkin workspace to fulfill this dependency.

Estimator Module

This module provides an interface to PEL to perform object recognition and pose estimation directly on the processed stream of point clouds. In order for PEL to correctly recognize objects a database of poses must be present in the database folder of PaCMaN Vision, the standard PaCMaN Ikea Object Database is already downloaded and unpacked into that folder for you, during the build process of PaCMaN Vision.

Estimator Module clusters object based on Euclidean point distance, thus objects cannot be too "near" or they would be clustered as a single object, likely ending in a failed pose estimation. The clustering tolerance parameter controls the maximum distance above which two group of points are considered two distinct clusters.

For each cluster found, Estimator runs a PEL progressive bisection algorithm, trying to estimate its pose. If it succeeds a green marker, representing the object mesh, is correctly aligned on the scene and it is published on PaCMaN Vision marker topics. Also its 6DOF pose with respect to the camera is broadcasted on ROS Tf topic. The options "publish markers" and "broadcast tf" control if the module needs to perform these two things or not.

Other options which control PEL algorithm behavior can be read in detail on the PEL wiki pages, in most scenarios the default parameters are just fine.

Once running, Estimator module starts a pose estimation only when the user call the service /pacman_vision/estimator/estimate or when he clicks the Gui button (which calls the service for you).

Tracker Module

This module takes an object from the pool of already estimated objects (from Estimator Module) and tries to track its pose in real time, while it's moving. Currently only one tracking at a time is supported.

Tracking is ICP-based, thus the less points are there to align, the faster the tracker is. It is almost mandatory to enable the downsampling filter from basic node, otherwise the Tracker won't be able to keep up with the constant stream of point clouds and it most likely fail. From my experience, a good value for downsampling leaf size is around 1.5cm - 2cm.

Once a Pose Estimation is performed a list of objects is populated on the Tracker Gui, selecting one of them enables the "Start Tracking" button, which calls the service "/pacman_vision/tracker/track_object", passing the object name selected as a request. The service starts the Tracker, you can tell that the tracker is tracking because the mesh marker of the tracked object turns from green to purple. Other markers are also being published and the current pose of the tracked object is broadcasted on the Tf ROS topic. Other Gui buttons control if the Tracker should publish these things or not.

A limitation of the Tracker is that it could get stuck on local minima (since it uses ICP), thus it could happen in certain situations, that the current object pose is not accurate, usually the Tracker resolves this situation by itself once the object is moved again.

To stop tracking call the service "/pacman_vision/tracker/stop_track" or press the corresponding Gui button (which calls the service for you), the mesh marker will turn green again and the object pose is updated to its last known transformation.

It could happen that the object moves outside the camera field, for example during a robot manipulation, if this happens the tracker will automatically pause and restart tracking as soon as some points are moving into the camera field of view. For this feature to work correctly you should take care on removing everything from the field of vision except the object you are tracking.

Please note that Tracker Module is still in experimental stage, thus its behavior could be changed/improved in future releases.

Estimator Module Summary		PACV_BUILD_RECOGNITION=1
/pacman_vision/estimator/estimate	service	Start a pose estimation procedure on last available frame of precessed scene.
/pacman_vision/markers	topic	Rviz Marker Array containing meshes of estimated objects, in green.
object poses	Tf	Broadcast poses of estimated objects with respect to the camera frame.
	dependency	Pose Estimation Library (PEL)

Tracker Module Summary		PACV_BUILD_RECOGNITION=1
/pacman_vision/tracker/track_object	service	Start tracking the object specified in the request.
/pacman_vision/tracker/stop_track	service	Stop tracking the currently tracked object.
/pacman_vision/markers	topic	Rviz Marker Array containing mesh of tracked object, in purple.
tracked pose	Tf	Broadcast the current pose of tracked object with respect to the camera frame.
	dependency	Pose Estimation Library (PEL)