RL Nodes

Status of the RL / path-planning nodes in ~/husky_ws (husky_rl, husky_control). Build rules are in ROS 2 Jazzy on the Onboard PC.

You need several terminals

The RL tests need 3–4 simultaneous SSH sessions to balto — one terminal per node, plus one to publish and one to echo. Do not try to do it all in one window: each node runs in the foreground and blocks its terminal, so you'll get confused.

Open each terminal with ssh [email protected], then in every one activate the venv and source the workspace (see ROS 2 Jazzy on the Onboard PC) before running anything.

Terminal	Purpose
1	Run a node (e.g. lambda_node)
2	Run another node (e.g. rl_policy)
3	Publish test input (ros2 topic pub ...)
4	Echo the output (ros2 topic echo ...)

Status

Node	Status
grid_geometry	Working — 11 unit tests pass
astar_node	Working — correct paths; all four goals mutually reachable; 23 waypoints from (1,1) to (15,9)
lambda_node	Working in fixed mode. CLI params need nested quotes (:="'false'") because they are declared as strings
rl_policy	Model loads, inference runs, and it receives lambda across transient-local QoS from a node that started earlier
point_controller	Blocked — crashes at construction (seven bugs, see below)

Blocked

• point_controller bugs (seven in total), including: an undeclared-parameter lookup, a method missing self, and publishing plain Twist to the wrong namespace (balto needs TwistStamped on /a200_0681/cmd_vel).
• No localization: UWB dropped, and GPS is not on the robot yet.
• cell_size_m unmeasured: A* emits grid integers while the controller expects metres.

The lambda anomaly

The policy takes an observation of agent_pos in [0,16]×[0,10] and lambda_values in [0,10], and outputs Discrete(4) — the goal index. All tests start from position (1,5):

lambda	expected goal	chosen
[9,1,1,1]	0 → (1,1)	0 → (1,1)	✅
[1,1,9,1]	2 → (1,9)	2 → (1,9)	✅
[1,1,1,9]	3 → (15,9)	1 → (15,1)	❌
[1,2,2,9]	3 → (15,9)	1 → (15,1)	❌

Ruled out: observation format, scaling, dtype and lambda range (the model's spaces were decoded and match what the node feeds exactly); a y-axis flip; and a general index scramble (indices 0 and 2 respond correctly and differ only in y).

What's left: index 3 looks unselectable — weighted at 9 twice and declined twice, with the policy falling back to index 1 both times.

Policy bias (random-lambda test)

With random lambdas drawn from 0–10 (the same range as training), the policy's choices are heavily skewed:

Goal	Position	Chosen
2	bottom-left (1,9)	41%
3	bottom-right (15,9)	8%

Lambda works in the right direction. Raising a goal's lambda pulls the policy toward it, so there is no sign error.

But the choice is decided by small biases. The Q-values for the four goals are nearly equal (within about 0.1), so small learned biases decide which goal wins. This also explains the lambda anomaly above: goal 3 is not strictly unselectable, it is just strongly disfavoured.

Possible causes (unconfirmed):

• Under-training — learning rate was only 5e-6.
• Uneven reward — the reward in adaptive_crl.py may treat the regions unevenly.

Next step: check the reward in adaptive_crl.py for asymmetry between goals, and retrain with a higher learning rate and compare.