User Manual - Forestry-Robotics-UC/fruc_dataset_apparatus GitHub Wiki

User Manual - Getting Started with FRUC Dataset Apparatus

Introduction

Welcome to the FRUC Dataset Apparatus - a professional-grade multi-sensor recording system designed for robotics research and dataset collection. This manual guides you through the complete workflow of setting up, configuring, and recording data.

What You'll Need

Hardware Requirements

Minimum System

  • CPU: Quad-core processor (8-core recommended)
  • RAM: 8 GB (16 GB recommended)
  • Storage: Fast SSD with at least 500 GB free space
  • USB ports: At least 3 USB 3.0 ports
  • Ethernet ports: 1 Gigabit Ethernet (for Ouster LiDAR)
  • Network interface: WiFi or Ethernet for system connectivity

Supported Platforms

  • Desktop workstations (Ubuntu 22.04/24.04)
  • Nvidia Jetson edge devices
  • SteamDeck (with Linux environment)
  • Server systems with Docker support

Required Sensors

The system is designed to work with these sensors (sensors are optional, add what you need):

  1. Xsens IMU - Inertial Measurement Unit

    • Connection: USB serial (460800 baud)
    • Data: 9-DOF IMU + magnetometer

  2. Intel RealSense D435i - Depth Camera

    • Connection: USB 3.0
    • Data: RGB video + depth maps

  3. Ouster LiDAR (OS1/OS2) - 3D LiDAR Scanner

    • Connection: Ethernet (link-local)
    • Data: Point clouds + reflectivity

  4. Emlid Reach - GNSS/GPS Receiver

    • Connection: Serial or Ethernet
    • Data: Position and velocity

Software Requirements

  • Docker 20.10 or later
  • Docker Compose 1.29 or later
  • Linux OS (Ubuntu 22.04 LTS recommended)
  • Git for repository management
  • kdialog (for interactive GUI menus)

Installation

Step 1: Install Docker and Docker Compose

On Ubuntu/Linux:

# Install Docker
sudo apt update
sudo apt install docker.io docker-compose

# Add your user to docker group (optional, for sudo-less operation)
sudo usermod -aG docker $USER
newgrp docker

# Verify installation
docker --version
docker-compose --version

For detailed instructions, visit: https://docs.docker.com/install/

Step 2: Clone the Repository

# Clone the main repository
git clone <repository-url> fruc_dataset_apparatus
cd fruc_dataset_apparatus

# If there are submodules, initialize them
git submodule init
git submodule update

Step 3: Install Udev Rules (for RealSense)

This step grants your user permission to access RealSense cameras without sudo:

cd realsense-udev-rules/
sudo bash realsense-udev-install.sh
cd ..

# Plug/unplug the RealSense camera for rules to take effect

Step 4: Set Serial Port Permissions (for Xsens)

# Check available serial ports
ls /dev/ttyUSB*

# Set permissions (replace ttyUSB0 with your device)
sudo chmod 666 /dev/ttyUSB0/

# Or run the provided script
sudo bash tty-perms.sh

Step 5: Verify Installation

Test that Docker containers build correctly:

cd docker/
docker-compose build --no-cache

# This will build all containers (may take 10-20 minutes on first run)

Configuration

Network Setup

For Ouster LiDAR (Link-Local Network)

The system needs a dedicated link-local network for Ouster:

# Create link-local network
docker network create --driver bridge --subnet=169.254.0.0/16 link-local-net

# Configure your Ethernet interface for link-local IP
# On Ubuntu:
sudo vim /etc/netplan/01-netcfg.yaml

Example netplan configuration:

network:
  version: 2
  ethernets:
    eth0:
      dhcp4: no
      addresses:
        - 169.254.50.1/16
      optional: true

Apply changes:

sudo netplan apply

Verifying Network Connectivity

# Check link-local network
ping 169.254.49.182  # Default Ouster hostname

# Or check DHCP on Ouster
arp-scan -l | grep ouster

Sensor Configuration

Xsens IMU

# Check USB serial connection
ls -la /dev/ttyUSB*

# Verify baud rate (460800 is standard)
cat /proc/tty/driver/usbserial

RealSense Camera

# Check if device is recognized
lsusb | grep RealSense

# Get serial number for reference
rs-enumerate-devices

Ouster LiDAR

# Edit docker_shared/ouster_config.yaml
nano docker/docker_shared/ouster_config.yaml

# Key parameters:
# - hostname: 169.254.49.182 (or your Ouster IP)
# - udp_dest: Set to your host machine's IP
# - lidar_port: Usually 8308
# - imu_port: Usually 8309

Emlid GNSS

# For serial connection, verify device
ls /dev/ttyUSB*

# For network connection, note the IP address
# Configure in launch script as needed

Recording Your First Dataset

Method 1: Interactive GUI Mode (Recommended)

This is the easiest way to record data, especially on desktop systems.

# From the repository root
./launch-system.sh

The system will present interactive dialogs:

Dialog 1: Recording Name

Choose a name for your recording. The system suggests a default:

Enter a name for the recording:
[2026-02-20_15-30-45__excited_maman-brigitte]

Press Enter to accept, or type a custom name. Names must be filesystem-safe (no special characters).

Dialog 2: Select Topics

A checklist appears with available ROS topics. Use:

  • [Space] to toggle topics on/off
  • [↑/↓] arrow keys to navigate
  • [OK] when finished

Recommended defaults (usually pre-selected):

  • ✓ IMU Data
  • ✓ Magnetometer
  • ✓ Ouster LiDAR Packets
  • ✓ RealSense Color
  • ✓ RealSense Depth
  • ✓ GPS/Fix
  • ✓ Transforms

Dialog 3: Bag Splitting

Choose how to split large recordings:

Option A: By File Size

  • Enters maximum size per bag file (in GB)
  • Example: 5 GB creates 5 GB chunks
  • Recommended: 5-10 GB per file

Option B: By Duration

  • Enters maximum duration per bag file (in seconds)
  • Example: 3600 creates 1-hour chunks
  • Recommended: 1800-3600 seconds

Dialog 4: Compression Profile

Choose compression strategy:

Profile Speed Compression Use Case
none Fastest None If fast storage available
fastwrite Very fast Medium Balanced, recommended
zstd_fast Fast Good Standard recording
zstd_small Slow Excellent Long-term storage

Recommended: zstd_fast (good balance)

Dialog 5: Confirmation

Review all settings. Click "Start Recording" to begin.

Method 2: SteamDeck (Podman)

If running on SteamDeck in Desktop Mode:

Step 1: Power On and Switch to Desktop

  1. Power on the SteamDeck
  2. Press the "STEAM" Physical button (bottom left front)
  3. Select "Power" → "Switch to Desktop"

Step 2: Launch the Recording System

  1. Open Terminal
  2. Navigate to the repository: 
    cd ~/fruc/fruc_dataset_apparatus/
  3. Run: ./launch-system.sh
    • The scripts automatically detect and use Podman on SteamDeck
    • Works exactly like the desktop version
  4. Follow the interactive dialogs (name, topics, splitting, compression)

Step 3: Control Recording with SteamDeck Buttons

Use the back buttons to start and stop recording:

To START Recording:

  • Long press the back left button (red circle "1")
  • Single click the back right button (green circle "2")

To STOP Recording:

  • Long press the back left button (red circle "1")
  • Single click the back top-left button (blue circle "3")

Step 4: Locate Your Recording

Recording files are saved to: ~/fruc/fruc_dataset_apparatus/rosbags/

Note: SteamDeck uses Podman (pre-installed) instead of Docker. The system automatically handles this - no special commands needed.

Method 3: GNOME Environment

On systems with GNOME desktop:

./gnome-launch-system.sh

Same dialogs as interactive mode, but with GNOME native styling.

Method 4: Manual Container Commands (Advanced)

For custom setups or headless servers.

For Desktop/Workstation (Docker):

cd docker/

# Start all containers
docker-compose up -d

# Start recording in separate terminal
docker exec ros2-apparatus-recording bash -c \
  "ros2 bag record -s mcap --topics /imu/data /camera/color/image_raw -o /rosbags/my_recording"

# Monitor recording
docker logs -f ros2-apparatus-recording

# Stop when finished
docker-compose down

For SteamDeck (Podman):

cd docker/

# Start all containers (swap 'docker' → 'podman')
podman-compose up -d

# Start recording in separate terminal
podman exec ros2-apparatus-recording bash -c \
  "ros2 bag record -s mcap --topics /imu/data /camera/color/image_raw -o /rosbags/my_recording"

# Monitor recording
podman logs -f ros2-apparatus-recording

# Stop when finished
podman-compose down

Docker vs Podman Commands:

Operation Docker Podman
Build images docker-compose build podman-compose build
Start containers docker-compose up -d podman-compose up -d
List containers docker ps podman ps
View logs docker logs podman logs
Execute command docker exec podman exec
Stop containers docker-compose down podman-compose down

Monitor recording

docker logs -f ros2-apparatus-recording

Stop when finished

docker-compose down


## During Recording

### Monitoring Data Quality

While recording is active, you can monitor in real-time:

```bash
# In a new terminal, check active topics
ros2 topic list

# Monitor specific topic frequency
ros2 topic hz /imu/data
ros2 topic hz /camera/color/image_raw

# View live topic data (use Ctrl+C to stop)
ros2 topic echo /imu/data --once

Visualization with Foxglove Bridge

Real-time visualization is available at IP:9092:

  1. Open web browser: http://localhost:9092
  2. Select "Open Connection"
  3. Enter ROS2 connection details (should auto-detect)
  4. Choose visualization type
  5. Add panels for different topics

Stopping Recording

Interactive Mode

The script will prompt you when ready to stop.

Manual Mode

# Find the recording container
docker ps | grep recording

# Stop gracefully (allow finishing current writes)
docker stop ros2-apparatus-recording

# Or force stop
docker kill ros2-apparatus-recording

# Stop all containers
./stop-system.sh

After Recording

Locating Your Data

Rosbags are saved in: ./rosbags/[timestamp]__[name]/

Example structure:

rosbags/
└── 2026-02-20_15-30-45__excited_maman-brigitte/
    ├── 2026-02-20_15-30-45__excited_maman-brigitte_0.mcap
    ├── 2026-02-20_15-30-45__excited_maman-brigitte_1.mcap
    ├── info.txt
    └── metadata.yaml

Checking Recording Quality

# Basic information
ros2 bag info rosbags/2026-02-20_15-30-45__excited_maman-brigitte/

# Check recorded topics
ros2 bag info -d rosbags/2026-02-20_15-30-45__excited_maman-brigitte/

# Verify data integrity
rosbag2_cpp_utilities validate rosbags/2026-02-20_15-30-45__excited_maman-brigitte/

Playback for Verification

# Play back recording (real-time)
ros2 bag play rosbags/2026-02-20_15-30-45__excited_maman-brigitte/ --loop

# Play at 2x speed
ros2 bag play rosbags/2026-02-20_15-30-45__excited_maman-brigitte/ --rate 2.0

# Seek to specific time
ros2 bag seek rosbags/2026-02-20_15-30-45__excited_maman-brigitte/ -s 30

Data Accessibility

Convert to Other Formats

# Convert MCAP to legacy rosbag format
rosbag2 convert -i rosbags/my_recording -o rosbags/my_recording_legacy

# Export to CSV (example for IMU)
ros2 bag convert -i rosbags/my_recording -f csv -o output.csv --topics /imu/data

# Export to ROS1 format (if needed)
rosbag2 convert -i rosbags/my_recording -f rosbag1 -o rosbags/my_recording_ros1

Share Recording

For sharing with other researchers:

# Compress the rosbag
tar -czf my_recording.tar.gz rosbags/my_recording/

# Get file size
du -sh my_recording.tar.gz

# Calculate checksum for verification
sha256sum my_recording.tar.gz > my_recording.sha256

Advanced Usage

Custom Topic Lists

Create a .env file to save your recording configuration:

# .env file in docker/ directory
TOPICS="
/imu/data
/imu/mag
/camera/color/image_raw
/ouster/points
/fix
"

COMPRESSION=zstd_fast
SPLIT_SIZE=5  # GB

Pre-Recording System Check

Run automated diagnostics:

cd docker/
bash pre_recording_test.sh

# This:
# 1. Starts all sensor containers
# 2. Records 7 minutes of test data
# 3. Validates sensor connectivity
# 4. Generates diagnostic report

Monitoring System Performance

For Desktop (Docker):

# Check container resource usage during recording
docker stats ros2-apparatus-recording

# Monitor CPU and memory
top -b -n 1 | head -20

# Check disk write speed
iotop -o

# Verify network connectivity to sensors
mtr 169.254.49.182  # Ouster

For SteamDeck (Podman):

# Check container resource usage during recording
podman stats ros2-apparatus-recording

# Monitor CPU and memory
top -b -n 1 | head -20

# Check disk write speed
iotop -o

# Verify network connectivity to sensors
mtr 169.254.49.182  # Ouster

Advanced Recording Options

Edit the recording command in launch-system.sh:

# Add these options to ros2 bag record:
--compression-queue-depth 10    # Increase for faster writes
--compression-threads 4         # Use 4 threads for compression
--storage-config-uri file://config.yaml  # Custom storage config

Troubleshooting

"Cannot connect to sensor" Error

Problem: One or more sensors not appearing in topics

Solutions:

  1. Check physical connections (USB, Ethernet cables)
  2. Verify device permissions: 
    ls -la /dev/ttyUSB*  # Should be readable by user
  3. Restart the sensor container: 
    docker-compose restart xsens  # (replace with sensor name)
  4. Check sensor power supply

"Insufficient disk space" Warning

Problem: Recording stops due to disk full

Solutions:

  1. Check available space: 
    df -h /path/to/rosbags
  2. Delete old recordings: 
    rm -rf rosbags/old_recording_*/
  3. Compress existing recordings: 
    tar -czf rosbags/old_recording.tar.gz rosbags/old_recording/
  4. Reduce compression ratio (use zstd_small)
  5. Move recordings to external storage

"Bag file corrupted" Error

Problem: Playback fails or info command fails

Solutions:

  1. Verify file integrity: 
    ros2 bag info rosbags/recording/ --verbose
  2. If using MCAP format, validate: 
    mcap-cli diagnose rosbags/recording/recording_0.mcap
  3. For partial recovery, use ros2 bag convert to another format

Topics Not Recording

Problem: Some topics appear in dialog but no data recorded

Solutions:

  1. Verify topic is actually publishing: 
    ros2 topic hz /topic_name
  2. Check message volume (if too slow): 
    ros2 topic rate /topic_name
  3. Ensure topic is selected in the dialog
  4. Check recording container logs: 
    # On Desktop (Docker):
docker logs ros2-apparatus-recording

# On SteamDeck (Podman):
podman logs ros2-apparatus-recording

Performance Issues (Dropped Frames, Lag)

Problem: Recording appears to lag or drop data

Solutions:

  1. Reduce number of recorded topics
  2. Use simpler compression (fastwrite or zstd_fast)
  3. Disable unnecessary containers: 
    # On Desktop (Docker):
docker-compose stop visualization  # Stop Foxglove if not needed

# On SteamDeck (Podman):
podman-compose stop visualization
  4. Check system resources: 
    # Desktop:
docker stats

# SteamDeck:
podman stats

# Both:
htop
  5. Close other heavy applications

Data Processing Examples

Extract Single Topic to CSV

python3 << 'EOF'
import sqlite3
from rosbag2_py import SequentialReader, StorageOptions

storage_options = StorageOptions(uri='rosbags/my_recording', storage_id='mcap')
reader = SequentialReader()
reader.open(storage_options)

with open('imu_data.csv', 'w') as f:
    f.write('timestamp,x,y,z,rx,ry,rz\n')
    while reader.has_next():
        topic, data, timestamp = reader.read_next()
        if topic == '/imu/data':
            # Extract data and write to CSV
            x = data.linear_acceleration.x
            # ... write to file
EOF

Create Point Cloud Video

# Use ROS tools to convert to PCD
ros2 bag convert -i rosbags/my_recording --topics /ouster/points -o pcd_output/

# Then use CloudCompare or PCL viewer
pcl_viewer pcd_output/*.pcd

Best Practices for Recording

Pre-Recording Checklist

  • All sensors physically connected
  • Serial/USB permissions configured
  • Network IP addressed (Ouster)
  • Sufficient disk space (5x your estimated needs)
  • Run pre-recording test: ./docker/pre_recording_test.sh
  • Check all topics appear: ros2 topic list
  • Start recording and verify initial data capture

Quality Guidelines

  1. Lighting: Ensure adequate lighting for camera (RealSense works best at 100-1000 lux)
  2. Motion: Smooth, deliberate movements (avoid rapid jerking)
  3. Coverage: Record from multiple viewpoints/positions for better reconstruction
  4. Duration: 2-5 minutes per scenario is typical
  5. Ground Truth: Consider markers for calibration if needed

Storage and Organization

datasets/
├── 2026-02-20_outdoor_daylight/
│   ├── recording_1/
│   └── recording_2/
└── 2026-02-21_indoor_fluorescent/
    └── recording_1/

Name recordings meaningfully for later identification.

Getting Help

If you encounter issues:

  1. Check the Troubleshooting Wiki Page
  2. Review Technical Architecture for system design
  3. Check Docker container logs: docker logs container_name
  4. Run diagnostics: ./docker/pre_recording_test.sh
  5. Contact the development team or check GitHub issues

Last Updated: 2026-03-30
For technical details, see: Technical Architecture
For advanced configuration, see: Developer Guide