Goal: classify the data-acquisition/generation methods 2024–2026 papers & companies propose for training policies on a humanoid five-finger dexterous hand, as a data pyramid (broad-abundant-far-from-hardware base → scarce-high-fidelity-on-hardware tip). Two hand-specific axes cut across every tier: the retargeting gap (human 5-finger → robot 5-finger) and force/tactile (the signal lost first as you descend). Companions: Dexterous Manipulation · Human Video → Robot Transfer · Tactile VLA · Cross-Embodiment.

flowchart TB
  L6["L6 · Target-hand teleop / real demos<br/>scarce · gold-standard · on-hardware"]
  L5["L5 · Simulation / synthetic grasp<br/>billion-scale · physics-labeled (force in sim)"]
  L4["L4 · Retargeting (human → robot 5-finger)<br/>the load-bearing bridge"]
  L3["L3 · Wearable glove / exoskeleton<br/>action-aligned finger kinematics + contact"]
  L2["L2 · Egocentric instrumented dex datasets<br/>dense hand-pose (+ some pressure)"]
  L1["L1 · Web / internet human-hand video<br/>unbounded · no action/force labels"]
  L6 --- L5 --- L4 --- L3 --- L2 --- L1
  T["cross-cut · TACTILE / FORCE (inject at L3/L5/L6)"] -.-> L4

Quantity grows downward; fidelity to the actual 5-finger hardware grows upward. The 2026 shift (see §5) is that the pyramid is inverting — human video (base) is becoming the primary data, with teleop reserved for the scarce tip.

Marks reflect training necessity, eval-only shown separately: ● required for training · ○ auxiliary (optional in training) · △ eval / deploy only — not training · blank = not used. Last column: is target-hand teleop/robot data used for fine-tuning? — Yes (+ volume) · eval-only · No. 🖐 = five-/multi-finger dexterous hand · 🤏 = gripper / 2-finger.

Reading it (papers): foundation-scale methods (EgoScale/UniDex/Being-H0/T-Rex) fine-tune on a target-hand teleop tip (● L6, "Yes"); device-free-video (Dexterous Point Policy, DO AS I DO) and capture-interface (DexUMI/DexEXO) methods keep the robot for eval/deploy only (△).

Fact-check: Genesis uses no simulation in training — glove+video only (>200k h); its Genesis-World sim is eval-only (L5 = △). Gripper-based programs (π, Dyna, AgiBot) are included but marked 🤏 — grippers/2-finger, not 5-finger dexterity; Dyna's DYNA-2 claims a 5-finger extension (unverified).

• 🆕 Genesis AI — GENE-26.5 (May 2026, Paris + San Carlos; Zhou Xian; vendor claims). A glove-first data engine: a tactile-e-skin glove giving a 1:1:1 mapping glove ↔ human hand ↔ robot hand (claimed 100× cheaper, 5× more data-efficient than teleop). Pretrained on real human data only — glove demos + egocentric + third-person/internet video (>200,000 h); no simulation in training (Genesis-World sim is evaluation-only, "zero simulation training data"). Then most tasks need < 1 h of task-specific robot data (< 200 episodes) for fine-tuning.
• 🆕 GR-Dexter (ByteDance Seed, Jan 2026). Explicit three-source pyramid — web VL + cross-embodiment real-robot (Fourier ActionNet ~140 h, OpenLoong 100k+, RoboMIND 107k) + 800 h human trajectories. ~20 h/task bimanual teleop (Meta Quest + MANUS) is used to train both the VLM backbone and the action DiT (21-DoF ByteDexter V2). → teleop = training, not eval.
• EgoScale (NVIDIA/Berkeley/UMD). 20,854 h human-video pretrain + a mid-training stage on aligned human-robot play (robot data used for training; +54% over no-pretrain; 22-DoF hand).
• 🆕 Dexterous Point Policy (KAIST, Jun 2026). The counter-example: trains only on human-video keypoints, no robot data; the robot appears only at eval (75% vs a VLA baseline's 1%) — the "zero-teleop-training" frontier.

Verdict on "is teleop used for fine-tuning?" — Yes, in every SOTA foundation stack, but as a small fine-tuning tip: Genesis < 1 h · DYNA-2 minutes–hours · GR-Dexter ~20 h/task · RLDX-1 small · EgoScale human-robot play · T-Rex 100 h. Only device-free-video and pure capture-interface methods keep robot data out of training (eval/deploy only). The 2026 achievement is collapsing the teleop fine-tuning tip toward ~1 hour, not eliminating it.

Incumbents (Figure, Tesla, AgiBot, Unitree, Fourier) are teleop/fleet-first — they scale L6 by industrializing collection (Figure's fleet mmWave offload, AgiBot's 4,000 m² facility, Tesla's factory floor); newcomers (Genesis, Dyna, RLWRLD) are video/glove-first — they shrink L6 to a sub-hour tip. Both converge on the same goal — make on-hand data cheaper — one by industrializing teleop, the other by replacing it with human video/glove. Note the geography: Chinese makers dominate the open large-scale teleop datasets (AgiBot World >1M, Unitree open full-body), while US labs (Figure, Tesla) keep fleet data proprietary. (Gripper/2-finger takeaway: UMI-style handheld grippers and parallel-jaw generalists — π, Dyna, AgiBot — win on data-collection speed but don't test in-hand five-finger dexterity.)

• L4 retargeting is the bottleneck. The abundant L1–L3 data only pays off if human→robot-hand mapping is faithful. 2026's move is dynamics/force-aware and calibration-free retargeting: DexMachina (decaying virtual-controller curriculum), Functional Force-Aware Retargeting, DO AS I DO (4D hand-object dynamics, 25%→71–81%), AnyDexRT (few-shot, calibration-free). Kinematic-only retargeting drops force and fails on contact-rich 5-finger tasks.
• Tactile/force is lost going down. L1–L2 have almost none, so contact-rich skills need force re-injected at L3 (glove contact), L5 (sim force), or L6 (real) — via DECO (plug-in tactile adapter), EgoTactile (pressure from video), T-Rex (tactile-reactive), and cross-sensor CTSRL.

The dominant production recipe for a humanoid 5-finger hand today is:

Teleoperation (MANUS glove / VR / exoskeleton) → retargeting → on-hand demos (L3 + L4 + L6), pretrained/augmented with simulation + synthetic grasps (L5).

• The industry splits two ways (§3c): hardware incumbents (Figure fleet-offload, Tesla factory data, AgiBot's 4,000 m² facility + >1M-trajectory AgiBot World, Unitree/Fourier open teleop datasets) are teleop/fleet-first; software newcomers (Genesis glove-1:1:1, Dyna/RLWRLD human-video) shrink L6 to a sub-hour tip. Chinese makers lead the open large-scale teleop datasets; US labs keep fleet data proprietary.
• ICRA-2026 vendors (MANUS, TESOLLO, ByteDexter 20-DoF) center glove-based dexterous teleop + software retargeting.
• Shared autonomy is the emerging cost-cutter: a human drives the arm (macro) via VR while an autonomous hand-VLA handles fine finger control (micro) — 2511.00139 — reducing operator burden that pure teleop can't scale past.
• Sim remains the safe, force-labeled bulk (Dex1B, DexGrasp-Zero, sim-to-real RL like DexNDM/RobustDexGrasp).
• Teleop's role is now a fine-tuning tip, not the whole dataset (§3b). The SOTA stacks pretrain on video/glove/sim, then fine-tune on a small on-hand teleop set — Genesis GENE-26.5 < 1 h, GR-Dexter ~20 h/task, DYNA-2 minutes–hours, T-Rex 100 h. Glove-first engines (Genesis's 1:1:1 tactile glove) push the "teleop replacement" further while still keeping a sub-hour fine-tune.

In short: teleop is still the quality anchor — but shrunk to a ≤1-hour fine-tuning tip; sim is the safe bulk; human video / glove is the new frontier base.

1. The pyramid inverts — human-video scaling laws become the base. EgoScale shows dexterous performance scales log-linearly (R²=0.9983) with human-video hours; DYNA-2 pushes to ~1M h; UniDex spans 8 hands. Teleop retreats to the scarce high-fidelity tip.
2. Device-free RGB video drops the glove: VideoManip / DO AS I DO reconstruct 4D hand-object dynamics from ordinary video — the cheapest possible L1→L4 path.
3. Calibration-free, operator-agnostic retargeting makes L4 universal: AnyDexRT (few-shot), DexEXO / YUBI (wearability-first / no-retarget interfaces). If L4 becomes cheap and faithful, the whole base transfers.
4. Tactile/force as a first-class tier, plus cross-sensor foundations (T-Rex, CTSRL, EgoTactile pressure-from-video) — recovering the signal the human-video base can't carry.
5. Cross-hand / morphology co-design — one pyramid, many hands: House of Dextra (co-optimize morphology + policy), One-Hand (canonical URDF), UniDex's 8-hand suite.
6. Unified foundation + a shrinking teleop tip — full-stack engines consume every tier and drive the on-hand fine-tune toward ~1 hour: Genesis GENE-26.5 (glove-first + < 1 h robot FT), GR-Dexter (three-source pyramid, ~20 h/task), RLDX-1, plus the WAM/MoT convergence (DYNA-2, Motus, Being-H0) trained across L1→L6 with one checkpoint; see VLA Hybrid Architectures.
7. The "zero-teleop-training" frontier — device-free-video methods that keep robot data out of training entirely (Dexterous Point Policy 2606.10614-style keypoints, DO AS I DO) — robot used only for eval. Whether these can match the sub-hour-fine-tune stacks is the open contest.

The through-line: whoever makes retargeting (L4) faithful-and-cheap and tactile (cross-cut) scalable will convert the abundant human-video/glove base into real 5-finger skill with a ≤1-hour teleop tip — those two bridges (plus the shrinking tip), not raw data volume, are the 2026–27 battleground.

• Companion reviews: Dexterous Manipulation · Human Video → Robot Transfer · Tactile VLA · Cross-Embodiment · Single-Checkpoint Multi-Robot
• Key data papers: EgoScale · UniDex · EgoDex · DexUMI · DexMachina · DexGrasp-Zero · DexNDM · Being-H0
• Retargeting deep-dives: DO AS I DO (2606.19333) · AnyDexRT (2607.08341)
• Capture-interface deep-dives: YUBI (2606.10244) · DexEXO (2603.17323)
• Tactile / industry deep-dives: T-Rex (2606.17055) · RLDX-1 (RLWRLD)
• Industry deep-dive: Genesis GENE-26.5 (Genesis AI, vendor)
• Humanoid-company data engines (§3c): Figure 03 · Tesla Optimus Gen-3 hand · AgiBot World (2503.06669) · Unitree Humanoid Everyday (2510.08807)
• Recent additions (no page yet): GR-Dexter (ByteDance, 2512.24210) · Dexterous Point Policy (KAIST, 2606.10614) · Shared-Autonomy (2511.00139)

← Back to Reviews · Home