Formal Robotics Specification – Camera Event and Lidar Sensor Event Correlation

1. System Overview

The system models a robotic perception stack that integrates lidar-based spatial mapping with QR code recognition events. The objective is to maintain a consistent representation of the environment and to correlate heterogeneous sensor events in time.

---

2. Label Marker Representation

The LabelMarker is a 2D occupancy structure.

Each marker stores the following state variables:

• label: the QR-code label
• location: the 2D location of label marker
• weight: the weight of signal (confidence)
• t_marker: timestamp of the last camera event related to this marker
• t_clean: timestamp of the clean marker

Each marker represents a local estimate of environmental occupancy derived from lidar observations and camera recognition event. All the label markers are grouped as a map of unique label id markers.

---

3. LiDAR Event Processing Model

3.1 Input Definition

A LiDAR event is defined as:

$$ L = (t, r, S) $$

where:

• t is the timestamp
• r is the sensor direction
• S is the set of measured returns

---

3.2 Update Function

Let:

• $M_i$: be an existing marker matching the recognized label
• $\hat{p}$: be the fused position estimate from LiDAR and camera
• $\lambda > 0$: temporal decay factor

Define:

$$ \alpha = e^{-\lambda (t - t_{marker})} $$

---

3.3 Marker Update Logic

Case 1 — Valid observation (matched marker)

If a marker is detected within the correlated camera–LiDAR angular sector:

$$ M_i.location \leftarrow (1-\alpha) M_i.location + \alpha \hat{p} $$

$$ M_i.weight \leftarrow 1 $$

$$ M_i.t_{marker} \leftarrow t $$

---

Case 2 — New marker

If label is not present in the map:

• create $M_i$
• initialize:
  • $location = \hat{p}$
  • $weight = 1$
  • $t_{marker} = t$

---

Case 3 — No observation

If no marker is detected:

• increment no_detection_counter

Define LiDAR change condition:

$$ |S_t - S_{t-1}| > \delta $$

If LiDAR state changes, store current scan context.

If:

$$ noDetectionCounter \geq \theta_{clean} $$

then:

• reduce remaining marker weights in cleaning region by $-\frac{t - t_{marker}}{\tau_{clean}} w$ clamping to 0, 1
• remove markers where $w = 0$

---

3.4 Receptive (Cleaning) Region Definition

The receptive region $R(t) \subset \mathbb{R}^2$ is the set of points in the planar workspace that are jointly observable by both camera and LiDAR at time $t$, expressed in the robot-centered reference frame.

3.4.1 Camera Field of View Constraint

Let:

• $p=(x,y)$ be a point in the robot-centered frame
• $θ(p)$ be the bearing angle of $p$
• $∥p∥$ be the Euclidean distance
• $r_C(t)$ is the current camera pointing direction
• $r_L(t)$ is the current LiDAR pointing direction (or beam angle)

3.4.2 Camera Field-of-View Constraint

A point is visible to the camera if:

$$ |θ(p)−r_C(t)| \le \frac{θ_{cam}}{2} $$

where:

• $θ_{cam}$ is the horizontal field of view of the camera

---

3.4.3 LiDAR Angular Consistency Constraint

A point is consistent with LiDAR sensing if:

$$ |θ(p)−r_L(t)| \le \frac{\Delta θ_{lidar}}{2} $$

where:

• $θ_{lidar}$ is the horizontal field of view of the LiDAR

---

3.4.4 Range Constraint

A point must lie within the effective sensing range:

$$ ∥p∥≤\min(d(t), d_{max}) $$

where:

• $d(t)$ is the instantaneous LiDAR measured distance along direction $θ(p)$
• $d_{max}$ is the maximum reliable sensing distance

3.4.5 Final Definition of the Receptive Region

The receptive region is defined as:

$$ R(t)={p∈R^2 \left| ∣θ(p)−r_C(t)∣≤\frac{θ_{cam}}{2} ∧ ∣θ(p)−r_L(t)∣≤\frac{Δθ_{lidar}}{2} ∧ ∥p∥≤\min(d(t), d_{max}) \right| } $$

4. Camera Event Model

4.1 Definition

A camera event is defined as:

C = (t, m, d)

where:

• t is the timestamp
• m is the recognized marker
• d is the marker direction relative to the camera frame

---

4.2 Temporal Correlation Constraint

Each camera event must be associated with a LiDAR event such that:

Let:

• t_p be the proximity event timestamp
• t_c be the camera event timestamp
• Δt_corr be the correlation window

A valid association satisfies:

t_p < t_c ≤ t_p + Δt_corr

---

5. Robot State Classification

The system defines a discrete state space for camera-proximity correlation:

5.1 Missing Camera Event

Occurs if:

• no camera event exists, or
• the camera event occurs before the proximity event

---

5.2 Valid Camera Event

Valid ⇔ (t_c > t_p) ∧ (t_c ≤ t_p + Δt_corr)

---

5.3 Invalid Camera Event

Invalid ⇔ t_c > t_p + Δt_corr

In this case, spatial correspondence cannot be reliably established.

---

6. System Assumptions

• Sensor timestamps are globally ordered or synchronized
• Lidar and camera events are processed in discrete time steps
• Geometric projections assume a calibrated and fixed sensor frame relationship

---

7. Version

Release: 0.26.0

---