CVPR 2026 HandX - Heungwoo/research GitHub Wiki

HandX — Scaling Bimanual Motion and Interaction Generation

Venue: CVPR 2026 Category: Bimanual / Hand Motion Generation Trend tag: Trend 1 Affiliations: UIUC + Specs Inc. + Snap Inc.

Approach diagram

flowchart LR
  MOCAP["bimanual mocap data"] --> CURATE["consolidated dataset"]
  LLM["LLM-generated descriptions"] --> CURATE
  CURATE --> MODEL["bimanual motion model"]
  MODEL --> GEN["bimanual interaction generation"]
  SCALE["scaling-law study"] -.-> MODEL

Problem

Bimanual motion datasets are fragmented across sources (different mocap setups, sparse text descriptions, inconsistent action representations). There has been no clean scaling-law study of bimanual motion generation, partly because there has been no unified dataset.

Method

Consolidate and filter existing datasets for quality, plus re-collect new mocap data targeting underrepresented bimanual interactions with detailed finger dynamics.
Decoupled annotation strategy: first extract representative motion features (contact events, finger flexion), then use LLM reasoning to produce fine-grained, semantically rich text descriptions — rather than captioning raw motion directly.
Benchmark both diffusion and autoregressive generators under multiple conditioning modes, and introduce new hand-focused evaluation metrics for dexterous motion quality.
Train models at different scales to characterize scaling behavior.

Results

The dataset and the scaling-law results — larger models trained on larger, higher-quality datasets produce more semantically coherent bimanual motion (clear scaling trends across both diffusion and autoregressive backbones).

Significance

HandX is the dataset foundation for bimanual motion / interaction at CVPR 2026. It continues the human-video / bimanual-transfer thread (e.g. ManipTrans) tracked in the CVPR 2025 survey. The clean scaling-law evidence is the most directly comparable to LLM-scaling claims in the bimanual-motion literature.