One Hand to Rule Them All — Canonical Representations for Unified Dexterous Manipulation

Venue: RSS 2026 (Manipulation session) · Authors: Zhenyu Wei, Yunchao Yao, Mingyu Ding — UNC Chapel Hill · arXiv: 2602.16712 · project Category: Cross-embodiment hand representation Trend tag: RSS 2026 thread 5 — hands go cross-embodiment

Compiled from the verified RSS 2026 abstract and the paper's Fig. 1.

Key figure

Figure 1 of the paper. Left: structurally diverse hands (human, multiple robot morphologies) map into the canonical hand representation — each hand becomes a parameter vector (the colored bar codes) plus a canonicalized geometry, shown as color-matched "canonical hands" below the originals. Center: the representation conditions a single cross-embodiment policy; the same grasp is rendered on an original hand and its canonical twin, illustrating that the canonical URDF preserves functional grasp behavior. Right: the zero-shot morphology wheel — one policy grasping across hands spanning 2–4 fingers and 8–16 joints, the visual form of the 81.9%-on-unseen-LEAP claim.

Problem

Dexterous policies assume a fixed hand; new kinematic/structural layouts break them. What the field lacks is a representation of hand-space itself.

Method

• Parameterized canonical representation of dexterous hand architectures: a unified parameter space (capturing morphological/kinematic variation for conditioning) + a canonical URDF format (standardizing the action space while preserving each hand's dynamic and functional properties).
• A VAE over the parameter space yields a compact latent manifold where interpolations between embodiments produce smooth, physically meaningful morphology transitions.
• A grasping policy conditioned on the canonical representation learns across structurally diverse hands.

Results (as reported)

• Validated via grasp-policy replay, VAE latent encoding, and cross-embodiment zero-shot transfer in sim and real.
• 81.9% zero-shot success on an unseen 3-finger LEAP hand.

Significance

The complementary half of the cross-hand pair with DexGrasp-Zero: where DexGrasp-Zero aligns hands anatomically (graph nodes), this canonicalizes them parametrically (URDF + latent manifold). The learnable morphology manifold additionally opens hand co-design — interpolate to a target embodiment before it exists (cf. the end-effector co-design thread in RSS 2026's Robot & Sensor Design session). A scalable foundation-layer candidate for Review-Dexterous-Manipulation's universal-hand agenda.

← RSS 2026 survey · Home