Running OpenVINS

Prerequisites

• Completed the IMU calibration with allan_variance_ros2 following the instructions in this Allan Variance ROS2
• Completed the Kalibr camera calibration following the instructions in the Kalibr Camera IMU Calibration

Running OpenVINS

These instructions are based on the OpenVINS guide, adapted to run inside the allan_variance_ros2 docker container (see steps to run here: devcontainer wiki) for use with the realsense camera.

To try OpenVINS, inside the container:

cd ~/ros2_ws
export MAKEFLAGS="-j 2"
colcon build --event-handlers console_cohesion+
source install/local_setup.bash

# Run rviz
ros2 run rviz2 rviz2 -d src/open_vins/ov_msckf/launch/display_ros2.rviz

# Run the simulation
ros2 run ov_eval plot_trajectories none src/open_vins/ov_data/sim/udel_gore.txt
ros2 run ov_msckf run_simulation src/open_vins/config/rpng_sim/estimator_config.yaml

To run the OpenVINS MSCKF Estimator, inside the container:

Source: https://docs.openvins.com/gs-tutorial.html

Multi-state Constraint Kalman Filter (MSCKF): https://docs.openvins.com/namespaceov__msckf.html

Example ros2 bag for https://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets: Link

Required files: These files need to be placed in src/open_vins/ov_msckf/config/<config_name> where the config_name is a unique folder name for your config files:

• estimator_config.yaml - Contains OpenVINS specific configuration files. Each of these can be overridden in the launch file.
  • The launch file is src/open_vins/ov_msckf/launch/subscribe.launch.py
  • This can be copied from an existing config folder and modified
  • In this file, we can set the number of cameras, window initialisation time etc
• kalibr_imu_chain.yaml - IMU noise parameters and topic information based on the sensor in the dataset. This should be the same as Kalibr's (see IMU Noise Calibration (Offline)).
  • This is one of the yaml files obtained from the camera imu calibration. Copy specific values into this file while maintaining the same format as the examples. The following are the mappings of values from the *calibration-results-imucam.txt file, only needed if using the scale-misalignment model for the imu, otherwise keep defaults:
    • C_gyro_i -> R_IMUtoGYRO
    • accelerometers: M -> Ta
    • gyroscopes: M -> Tw
    • gyroscopes: A -> Tg
  • Note that %YAML:1.0 needs to be at the top of the file
• kalibr_imucam_chain.yaml - Camera to IMU transformation and camera intrinsics. This should be the same as Kalibr's (see Camera Intrinsic Calibration (Offline)).
  • This is the other yaml file obtained from the camera imu calibration, copy this file directly
  • Note that %YAML:1.0 needs to be at the top of the file as in the existing config examples

For running the node in ros2, use the src/open_vins/ov_msckf/launch/subscribe.launch.py file which will pick up configs in the src/open_vins/ov_msckf/config/ directory. To this file, we can pass the config parameter for the name of the config folder and pass the max_cameras:=1 flag to run monocular estimation (one camera only).

cd ~/ros2_ws
ros2 run rviz2 rviz2 -d src/open_vins/ov_msckf/launch/display_ros2.rviz

# Run the node in another terminal
cd ~/ros2_ws
export MAKEFLAGS="-j 2"
colcon build --event-handlers console_cohesion+
source install/local_setup.bash
ros2 launch ov_msckf subscribe.launch.py config:=<config_name>

# Play the data in another terminal or launch the realsense node for live data
ros2 launch realsense2_camera rs_launch.py enable_accel:=true enable_gyro:=true unite_imu_method:=2 enable_infra1:=true enable_infra2:=true depth_module.infra_profile:=848x480x30 rgb_camera.color_profile:=1280x720x30

ros2 bag play V1_01_easy # replace with bag/mcap file of data

Some of the topics published by the MSCKF node are:

Topic	Message Type
/ov_msckf/odomimu	nav_msgs/msg/Odometry
/ov_msckf/pathimu	nav_msgs/msg/Path
/ov_msckf/poseimu	geometry_msgs/msg/PoseWithCovarianceStamped
/ov_msckf/loop_feats	sensor_msgs/msg/PointCloud
/ov_msckf/points_slam	sensor_msgs/msg/PointCloud2

Notes on Estimator Configuration

The estimator_config.yaml file contains the configuration for the estimator placed in src/open_vins/ov_msckf/config/<config_name>. The following are some notes on the parameters in this file. A sample config file for the Intel Realsense D435i camera is provided in estimator_config.yaml.

• max_clones: The maximum number of clones to keep in the state vector. Increasing it allows more clones in the sliding window, but at the cost of increased memory usage and computation time.
• max_slam: The number of features to keep in the state vector. Increasing it allows more features to be tracked, but at the cost of increased memory usage and computation time.
• try_zupt: Whether to try to use zero velocity update (ZUPT) to improve the state estimate. This is a boolean value, set to true to enable ZUPT. it improves initialisation and tracking stability, by reducing the drift while the camera is stationary.
• init_window_time: How many seconds of data to use for initialisation.
• init_imu_thresh: The threshold for motion detection during initialization. Decreased for moresensitive motion detection.
• init_max_features: The maximum number of features to track during initialization. Increase for better initialization, more features to track during initialization.
• num_pts: The number of points per camera used in feature tracking.
• fast_threshold: This is the threshold for fast extraction. Lower means more features, but is more computationally expensive.
• grid_x, grid_y: Extraction sub-grid count for horizontal and vertical direction.
• min_px_dist: This is the distance between features, which should be lowered as more feature points are added. Note that although this can be decreased, features near each other provide less information.
• track_frequency: This is the frequency at which features are tracked, in frames/second.