Point_Cloud - RicoJia/notes GitHub Wiki

========================================================================

========================================================================

1. [x,y,z], from Lidar, SfM, CAD, RGBD Camera, Depth from images. Iphone has RGBD for face recognition (so cannot be fooled by image)

   

   • Perception: object detection, classification
   • 语义分割: how to segment a group of point clouds from another, and interpret them.
   • Also need to calibrate Lidar and RGB camera.
   • How to represent a 3D?
     • use triangles and put them in meshes. But triangles are complex (in older games before)

       

     • Voxel grid: wasting a lot of empty cells
     • Octree: can represent an object when there is one, but quite complex.

       

2. Challenges:

   1. Sparsity: point clouds far away will not be recognized properly
   2. Irregular: Not like pixels in a picture. No guarantee that you'll find points within a radius
   3. Lack of texture. 3 ppl walking side by side might be recognized as a car.
   4. Unordered: will confuse deep nets, just by swapping two examples.

      

   5. Rotation invariance for recognition

3. Methods

   1. classical: explanable: controllable. But hard to model semantics.

4. SLAM: loop closure, registration? what is registration?

5. Great books:

   1. probablistic graphical models, principle and techques
   2. Pattern recognition and machine learning (christopher bishop)
   3. Machine Learning, a probablistic perspective
   4. Computer Vision: models, learning and interference.
   5. Bayesian Reasoning and machine learning. (David barber)
   6. Deep learning: simple, object recognition is much better. But it's a black-box. Requires GPU, FPGA, and more difficult to find a job LOL
      • Data collection, date labelling, data training.

========================================================================

========================================================================

1. open3d

   • numpy -> open3d.PointCloud

     # Pass xyz to Open3D.o3d.geometry.PointCloud and visualize
     pcd = o3d.geometry.PointCloud()
     pcd.points = o3d.utility.Vector3dVector(xyz)
     o3d.io.write_point_cloud("../../TestData/sync.ply", pcd)

   • open3d.PointCloud -> numpy

     import open3d as o3d
     pcd_load = o3d.io.read_point_cloud("../../TestData/sync.ply")
     o3d.visualization.draw_geometries([pcd_load])
     
     # convert Open3D.o3d.geometry.PointCloud to numpy array
     xyz_load = np.asarray(pcd_load.points)```
     [[-3.00000000e+00 -3.00000000e+00 -3.89817183e-17]

     • make sure no inf in the point cloud

   • orientation: red is +x, green is +y. In default view, z is pointing outwards