Venue: CoRL 2025 (Best Paper Finalist) · Authors: Mengda Xu, Han Zhang, Yifan Hou, Zhenjia Xu, Linxi Fan, Manuela Veloso, Shuran Song — Stanford / Columbia / J.P. Morgan AI Research / CMU / NVIDIA · arXiv: 2505.21864 Category: Dexterous Manipulation / Data Trend tag: Human hand as a universal interface

flowchart LR
  H[Human wears<br/>hand exoskeleton<br/>+ tactile sensors] --> D[Teleop-free<br/>data collection]
  D --> TRAJ[Finger kinematics +<br/>tactile/haptic traces]
  TRAJ --> HW[Hardware adaptation:<br/>exoskeleton bridges<br/>human→robot kinematics]
  TRAJ --> SW[Software adaptation:<br/>inpaint robot hand<br/>over human hand in video]
  HW --> POL[Dex manipulation<br/>policy]
  SW --> POL

Collecting dexterous-hand data is painful: teleop rigs are brittle, motion-capture is indoor-only, and humans naturally have different hand morphology from any robot gripper. You want hours of diverse dex demonstrations but there's no scalable recorder.

DexUMI makes the human hand itself the universal manipulation interface via two complementary adaptations that close the human↔robot embodiment gap:

• Hardware adaptation: a wearable hand exoskeleton (not a passive glove) constrains the human hand to motions the target robot hand can physically execute, bridging the kinematics gap. Tactile sensors on the exoskeleton capture contact interactions and provide direct haptic feedback to the wearer during data collection.
• Software adaptation: high-fidelity robot-hand inpainting replaces the human hand with the robot hand in the recorded video, so the resulting demonstrations look as if the robot collected them — removing the visual embodiment gap for the policy.

No teleop and no mocap room are required. Validated on two dexterous robot hand platforms (Inspire Hand and XHand).

Best Paper Finalist. Across real-world tasks on both robot platforms, DexUMI achieves an average task success rate of 86%. Ablations show that tactile input and the software inpainting pipeline are each critical (large drops without them), and compare absolute vs. relative finger-action trajectories.

Paired with the Best Student Paper on humanoid control from video, DexUMI cements wearable human capture as the scalable dex-data strategy. Slots into ICLR 2026's dex thread via DexNDM, UniHM, RFS, and EgoDex.

• arXiv: https://arxiv.org/abs/2505.21864

• Visual Imitation → Humanoid
• Survey (CoRL 2025)

← Back to CoRL-2025