HiWET: Hierarchical World-Frame End-Effector Tracking for Long-Horizon Humanoid Loco-Manipulation

Venue: RSS 2026 (Sydney, Jul 13–17) · Session: Humanoids · paper #30 Authors: Zhanxiang Cao, Liyun Yan, Yang Zhang, Sirui Chen, Jianming Ma, Tianyue Zhan, Shengcheng Fu, Yufei Jia, Cewu Lu, Yue Gao arXiv: 2602.06341 · program page

Summary compiled from the arXiv paper (v1); all numbers quoted from the paper. Trend context: RSS 2026 survey.

Figure 1: (a)-(c) whole-body redundancy exploitation for extreme reaching poses (lowest/farthest, highest, outermost in semi-squat), each shown in simulation and on the real robot; (d) sim-to-sim world-frame trajectory tracking in MuJoCo (red target circle vs. green actual); (e)-(f) real-world long-exposure shots of square and circular trajectories traced by an LED on the end-effector.

Problem

Humanoid loco-manipulation commands are usually expressed in body-centric frames, so cumulative base drift and high-frequency oscillations from legged locomotion directly corrupt end-effector precision in the world frame; motion-imitation methods optimize joint-space rather than task-space error and need dense references. When targets leave the static reachable workspace, the robot must actively transport its base — a coupling body-centric formulations do not address.

Method

HiWET reformulates the task as world-frame end-effector tracking with a hierarchical RL scheme on the 29-DoF Unitree G1 (12 leg, 14 arm, 3 waist DoF): a high-level world-frame command policy plans in global coordinates, emitting subgoals (base velocity, body height, local end-effector targets plus an arm-activation mask), while a high-frequency low-level whole-body tracking policy converts them to joint commands under stability constraints. A Kinematic Manifold Prior (KMP) — a ResNet trained on an importance-sampled dataset (filtered by IK error and manipulability, derived from AMASS retargeting) — supplies kinematically valid upper-body references so the policy learns only residual corrections. Training runs in Isaac Lab; on hardware, global pose comes from a head-mounted Livox Mid-360 LiDAR + IMU via Fast-LIO2 with 10 Hz base updates.

Results

Low-level command tracking (Table I): HiWET achieves 12.4 ± 2.4 mm hand Cartesian error vs. 25.2 mm without KMP and 23.0 mm without the state estimator, and reduces base linear-velocity RMSE by roughly 20% vs. the HOMIE baseline. On world-frame geometric trajectories (star, heart, circle, spiral, rectangle initialized up to ±5 m away, success = mean error < 20 mm), HiWET attains the highest success rates and errors below 5 mm on some patterns; base repositioning to 5 m targets yields 0.101 m mean final position error, the lowest among variants. KMP-L (7.38M params) is over 5x faster than the PyRoki IK solver at single-sample inference (millisecond-level at batch 4000) with median position error below 15 mm and orientation error below 5 degrees. Zero-shot real-world tracking RMSE: 0.012 m (circle) and 0.015 m (square).

Significance

Makes the world frame — not the drifting body frame — the contract between planning and control, a prerequisite for long-horizon humanoid manipulation pipelines that hand VLA-style planners a spatial interface. Related threads: Review-Humanoid-VLA · Review-System-0-1-2.

← Back to RSS 2026 survey · RSS-2026-Papers · Home