Point Cloud-based Object localisation:¶
To determine the exact location of graspable objects around the MIRTE Master, the data gathered and transmitted by the camera must be interpreted. One effective method for achieving this is through the use of point clouds. As the name suggests, point clouds are collections of points that represent a three-dimensional space. These points may contain a variety of information, but the most important characteristic of a point during this project is its position, represented by its x, y, and z coordinates. During this project, the Open3D library was used to process the point cloud. This particular library has been selected because of its user-friendly API and popularity over PCL Zhou et al., 2018.
Detecting planes will be done using the Random Sample Consensus model, or RANSAC for short. This algorithm picks a number of points from the point cloud at random and fits a plane through them. It then evaluates which points in the point cloud are within the given distance from the plane Fischler & Bolles, 1981. With this method, point clouds can easily and quickly be filtered such that only the important parts, namely the clusters of points representing small objects on the ground, remain.
Next, an algorithm for discovering clusters is used called DBSCAN (Density-Based Spatial Clustering of Applications with Noise). The model evaluates the remaining points in the point cloud and determines the cluster to which they belong Ester et al., 1996. Afterwards, the identified clusters can be converted into bounding boxes, deciding an object’s position and orientation, as well as its dimensions.
Below, figures Figure 1 and Figure 2 show the input to output the constructed point cloud pipeline.
Figure 1:The point cloud as received from the camera topic
Figure 2:The resulting bounding boxes
Any objects that are in the costmap are excluded, resulting in only these bounding boxes remaining.
- Zhou, Q.-Y., Park, J., & Koltun, V. (2018). Open3D: A modern library for 3D data processing. arXiv Preprint arXiv:1801.09847.
- Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24(6), 381–395. 10.1145/358669.358692
- Ester, M., Kriegel, H.-P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, 226–231.