HAIC: Humanoid Agile Object Interaction Control via Dynamics-Aware World Model

Venue: RSS 2026 (Sydney, Jul 13–17) · Session: World Models & Memory · paper #13 Authors: Dongting Li, Xingyu Chen, Qianyang Wu, Bo Chen, Sikai Wu, Hanyu Wu, Guoyao Zhang, Liang Li, Mingliang Zhou, Diyun Xiang, Jianzhu Ma, Qiang Zhang, Renjing Xu arXiv: 2602.11758 · program page

Summary compiled from the arXiv paper (v2); all numbers quoted from the paper. Trend context: RSS 2026 survey.

Figure 1: a Unitree G1 humanoid performing the three interaction classes HAIC enables — (a) underactuated object interaction (skateboarding, cart pushing/pulling), (b) sequential interaction (pick up a box, load it onto the cart, then drive both forward in one policy), and (c) multi-terrain interaction (carrying a box over platforms, slopes, and stairs).

Problem

Humanoid human-object-interaction work mostly handles fully actuated objects rigidly coupled to the end-effector; underactuated objects like skateboards and carts have independent dynamics and non-holonomic constraints, generate inertial coupling forces, and frequently occlude the robot's onboard sensors. Existing approaches either rely on impractical external state estimation (LiDAR, markers, depth/RGB object tracking) or, lacking high-order dynamics modeling, suffer response lag during high-speed interactions.

Method

HAIC is a proprioception-centric framework built around a dynamics-aware world model (DWM) with three parts: an Object Adapter that predicts the object's relative position, orientation, linear/angular velocities and accelerations purely from proprioceptive history plus future reference motion; an Explicit Geometric Projection step that transforms a nominal canonical point cloud by the predicted pose, giving the policy a spatially grounded dynamic-occupancy representation of the (possibly invisible) object; and a Privilege Adapter that fuses this with proprioception into a predicted privileged feature. Training is a two-stage teacher-student pipeline: Stage 1 trains a privileged PPO teacher while warm-starting the student and world model via KL distillation; Stage 2 performs asymmetric fine-tuning where the student explores with a privileged critic and the world model keeps adapting to the student's distribution, stabilized by an EMA copy of world-model weights. Deployment on a Unitree G1 uses only onboard proprioception (joints, IMU), reference motion, and the nominal point cloud — no external sensing.

Results

Against HDMI* (a proprioception-only variant of HDMI retrained in the same setup), real-world skateboarding: HAIC reaches 100% glide / 60% complete success vs. 20% / 0% for the baseline (Table I). Cart manipulation: 100% success on both Pull and Push vs. 40% and 0% (Table II). Sequential cart-with-box tasks where the baseline scores 0%: HAIC gets 100% (Push Cart w/ Box) and 40% (Pull Cart w/ Box) (Table III). On multi-terrain carrying, HAIC succeeds on the slope-stair composition where the baseline fails (Table IV), and the paper reports a 100% success rate on blind multi-terrain box carrying. Ablations show a variant without acceleration prediction (Vec-Pose) can balance on the skateboard but fails to dismount.

Significance

A world model used not for visual prediction but as an internal dynamics estimator that turns a partially observed humanoid-object system into a controllable one — a distinctive entry in the World Models & Memory session. Related threads: Review-World-Models · Review-VLA-Memory.

← Back to RSS 2026 survey · RSS-2026-Papers · Home