2. Introduction - mclumd/baxter_pcl GitHub Wiki
Recognition and location of objects is an essential component for any cognitive science researcher/practitioner to test the practical approachability of their research. Before the advent of 3d scanners (RGB-D cameras), practitioners often manipulated the 2d image and extrapolated the object to appear as a 3d structure, and the distance or location of the object was obtained through the IR sensor obstructing the infrared rays. Both the tasks were independent of each other and had their own subset of problems in the real world. Later the invention of 3d scanners narrowed the problems by incorporating two independent tasks into a single one. The depth image of an RGB-D camera is given by a pattern of points(obtained from infrared rays objected by all the objects from the environment) along with the colored image. Further challenges of detecting, recognizing and locating objects from the huge point set is where the world stands to address. This document attempts to incorporate such a problem in a simple blocks-world environment with the help of the point cloud library (PCL). The environment is set in a closed room containing five colored blocks on a table along with some chairs and computers. This document describes an approach to obtain the shape of the 3d blocks and their point location by purging the other points from the environment in a six stage process.
METHOD
The 3d scanner mounted on the Baxter robot continuously publishes the point cloud of the scene (environment) onto a ROS topic named “/rgb/depth_registered/points” in the pcd format (x y z RGB). These scene cloud points using the point cloud library undergo a five phase process. In the first phase, filtering is done to limit the points to only a certain distance from the camera, thus restricting the view to the table along with the blocks. In the second phase, plane segmentation identifies the points of the plane from the point cloud and removes them from the scene cloud. While in the third phase we identify the points of the blocks by defining the possible hue ranges of the blocks. Towards the fourth phase, obtained points are clustered into groups using Euclidean clustering. This step recognizes if there are blocks of the same color or not. The last phase determines whether the obtained points belong to the shape of the blocks or not by using some mathematical constraints. These phases are individually explained in the latter part of the document.