Camera‐Lidar Calibration - ashBabu/Utilities GitHub Wiki
Overview
This is done using the package direct_visual_lidar_calibration. An excellent tutorial is available here. This is a target-less method which means a rosbag with rich features and the camera and lidar have overlapping field of view is used. I am still listing out all the steps once again and some minor changes required.
Installation
From here
Main changes from what is given
- Eigen is used for Ubuntu 22 and above
- apt install of ceres
# Install dependencies
sudo apt install libomp-dev libboost-all-dev libglm-dev libglfw3-dev libpng-dev libjpeg-dev
# Install GTSAM
git clone https://github.com/borglab/gtsam
cd gtsam && git checkout 4.2a9
mkdir build && cd build
cmake .. -DGTSAM_BUILD_EXAMPLES_ALWAYS=OFF \
-DGTSAM_BUILD_TESTS=OFF \
-DGTSAM_WITH_TBB=OFF \
-DGTSAM_USE_SYSTEM_EIGEN=ON \
-DGTSAM_BUILD_WITH_MARCH_NATIVE=OFF
make -j$(nproc)
sudo make install
# Install Ceres
sudo apt install libceres-dev
# Install Iridescence for visualization
git clone https://github.com/koide3/iridescence --recursive
mkdir iridescence/build && cd iridescence/build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
sudo make install
cd ~/ros2_ws/src
git clone https://github.com/koide3/direct_visual_lidar_calibration.git --recursive
cd .. && colcon build --symlink-install --cmake-clean-first --cmake-args -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-DNDEBUG"
The above colcon build arguments are necessary. Else, there might be an Eigen assertion error
Download the dataset livox.tar.gz using the link above and unzip it.
ls livox
# rosbag2_2023_03_09-13_42_46 rosbag2_2023_03_09-13_44_10 rosbag2_2023_03_09-13_44_54 rosbag2_2023_03_09-13_46_10 rosbag2_2023_03_09-13_46_54
$ ros2 bag info livox/rosbag2_2023_03_09-13_42_46/
# Files: rosbag2_2023_03_09-13_42_46_0.db3
# Bag size: 582.9 MiB
# Storage id: sqlite3
# Duration: 15.650s
# Start: Mar 9 2023 13:42:46.665 (1678336966.665)
# End: Mar 9 2023 13:43:02.316 (1678336982.316)
# Messages: 2972
# Topic information: Topic: /livox/points | Type: sensor_msgs/msg/PointCloud2 | Count: 157 | Serialization Format: cdr
# Topic: /livox/imu | Type: sensor_msgs/msg/Imu | Count: 2597 | Serialization Format: cdr
# Topic: /livox/lidar | Type: livox_interfaces/msg/CustomMsg | Count: 157 | Serialization Format: cdr
# Topic: /image | Type: sensor_msgs/msg/Image | Count: 30 | Serialization Format: cdr
# Topic: /camera_info | Type: sensor_msgs/msg/CameraInfo | Count: 31 | Serialization Format: cdr
Preprocessing
# -a : Detect points/image/camera_info topics automatically
# -v : Enable visualization
ros2 run direct_visual_lidar_calibration preprocess livox livox_preprocessed -av
After running preprocess, you can find a directory named livox_preprocessed that containts generated dense point clouds, camera images, and some meta data (screenshot):
$ ls livox_preprocessed/
# calib.json rosbag2_2023_03_09-13_44_10_lidar_intensities.png rosbag2_2023_03_09-13_44_54.png rosbag2_2023_03_09-13_46_54_lidar_intensities.png
# rosbag2_2023_03_09-13_42_46_lidar_indices.png rosbag2_2023_03_09-13_44_10.ply rosbag2_2023_03_09-13_46_10_lidar_indices.png rosbag2_2023_03_09-13_46_54.ply
# rosbag2_2023_03_09-13_42_46_lidar_intensities.png rosbag2_2023_03_09-13_44_10.png rosbag2_2023_03_09-13_46_10_lidar_intensities.png rosbag2_2023_03_09-13_46_54.png
# rosbag2_2023_03_09-13_42_46.ply rosbag2_2023_03_09-13_44_54_lidar_indices.png rosbag2_2023_03_09-13_46_10.ply
# rosbag2_2023_03_09-13_42_46.png rosbag2_2023_03_09-13_44_54_lidar_intensities.png rosbag2_2023_03_09-13_46_10.png
# rosbag2_2023_03_09-13_44_10_lidar_indices.png rosbag2_2023_03_09-13_44_54.ply rosbag2_2023_03_09-13_46_54_lidar_indices.png
Initial guess (Manual)
ros2 run direct_visual_lidar_calibration initial_guess_manual livox_preprocessed
- Right click a 3D point on the point cloud and a corresponding 2D point on the image
- Click Add picked points button
- Repeat 1 and 2 for several points (At least three points. The more the better.)
- Click Estimate button to obtain an initial guess of the LiDAR-camera transformation
- Check if the image projection result is fine by changing blend_weight
- Click Save button to save the initial guess
Fine registration
Perform NID-based fine LiDAR-camera registration to refine the LiDAR-camera transformation estimate:
ros2 run direct_visual_lidar_calibration calibrate livox_preprocessed
Calibration result file
Once the calibration is completed, open livox_preprocessed/calib.json with a text editor and find the calibration result T_lidar_camera: [x, y, z, qx, qy, qz, qw] that transforms a 3D point in the camera frame into the LiDAR frame (i.e., p_lidar = T_lidar_camera * p_camera).
calib.json also contains camera parameters, manual/automatic initial guess results (init_T_lidar_camera and init_T_lidar_camera_auto), and some meta data.
calib.json
{
"camera": {
"camera_model": "plumb_bob",
"distortion_coeffs": [
-0.04203564850455424,
0.0873170980751213,
0.002386381727224478,
0.005629700706305988,
-0.04251149335870252
],
"intrinsics": [
1452.711762456289,
1455.877531619469,
1265.25895179213,
1045.818593664107
]
},
"meta": {
"bag_names": [
"rosbag2_2023_03_09-13_42_46",
"rosbag2_2023_03_09-13_44_10",
"rosbag2_2023_03_09-13_44_54",
"rosbag2_2023_03_09-13_46_10",
"rosbag2_2023_03_09-13_46_54"
],
"camera_info_topic": "/camera_info",
"data_path": "livox",
"image_topic": "/image",
"intensity_channel": "intensity",
"points_topic": "/livox/points"
},
"results": {
"T_lidar_camera": [
0.023215513184544914,
-0.049304803782681345,
-0.0010268378243773314,
0.002756788930227678,
0.7121675520572427,
0.0038417302647440915,
0.7019936032615696
],
"init_T_lidar_camera_auto": [
0.01329274206061581,
-0.055999414521382934,
0.0033183505131586903,
0.002471267432195032,
0.7121558216581672,
0.0030750358632291534,
0.7020103437059168
]
}
}