Point Cloud-Based Object Localization¶
To determine the exact location of graspable objects around the MIRTE Master, the data gathered and transmitted by the camera must be interpreted. One effective method for achieving this is through the use of point clouds. As the name suggests, point clouds are collections of points that represent a three-dimensional space. These points may contain a variety of information, but the most important characteristic of a point during this project is its position, represented by its x, y, and z coordinates. During this project, the Open3D library was used to process the point cloud. This particular library has been selected because of its user-friendly API and popularity over PCL Zhou et al., 2018.
Planes are detected using the Random Sample Consensus (RANSAC) algorithm. This algorithm picks a number of points from the point cloud at random and fits a plane through them. It then evaluates which points in the point cloud are within the given distance from the plane Fischler & Bolles, 1981. With this method, point clouds can easily and quickly be filtered such that only the important parts, namely the clusters of points representing small objects on the ground, remain.
Next, a clustering algorithm called DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used. The algorithm evaluates the remaining points in the point cloud and determines the cluster to which they belong Ester et al., 1996. Afterwards, the identified clusters can be converted into bounding boxes, determining an object’s position and orientation, as well as its dimensions.
Below, figures Figure 1 and Figure 2 show the input and output of the point cloud pipeline.
Figure 1:The point cloud as received from the camera topic
Figure 2:The resulting bounding boxes
Any objects that are part of the costmap are excluded, leaving only these bounding boxes.
- Zhou, Q.-Y., Park, J., & Koltun, V. (2018). Open3D: A modern library for 3D data processing. arXiv Preprint arXiv:1801.09847.
- Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24(6), 381–395. 10.1145/358669.358692
- Ester, M., Kriegel, H.-P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, 226–231.