Paper: "T-Rex: Tactile-Reactive Dexterous Manipulation" — arXiv 2606.17055 (Jun 15 2026) Authors: 34 authors incl. Dantong Niu, Zhuoyang Liu, Zekai Wang (affiliations not listed in preprint metadata) What it is: a variable-rate Mixture-of-Transformers VLA that runs a fast tactile expert alongside a slow visuomotor expert for reactive, force-controlled dexterity — the tactile-first apex of the Dexterous-Hand Data Pyramid (L6 teleop + tactile cross-cut).

Companions: Dexterous-Hand Data Pyramid · Tactile VLA · VLA Hybrid Architectures · OmniVTA (the visuo-tactile-WM counterpart) · Dexterous Manipulation.

1. Two experts at two rates. T-Rex splits flow-matching denoising into a slow Action Expert (visuomotor planning, τ ∈ [0.4, 1], vision-language context) and a fast Tactile Expert (τ ∈ [0, 0.4], real-time tactile only) — a variable-rate Mixture-of-Transformers so touch can refine actions without throttling the VLA.
2. Temporal tactile VQ-VAE. High-frequency touch is encoded by compressing 16-frame force histories per finger into discrete tokens (1D temporal conv, two strided downsamples); the current force vector bypasses compression for instantaneous contact; deformation maps → frozen ResNet-18 → 128-dim per fingertip.
3. Real hardware, real rates. Dexmate Vega-1 bimanual with two Sharpa Wave 22-DoF hands; 5 fingertip tactile sensors/hand (6-axis wrench + deformation depth); 300 Hz low-level loop, policy async at ~30 Hz.
4. +30 points on delicate tasks. 65% average across 12 tasks emphasizing delicate force control and deformable objects — vs the strongest baseline (EgoScale, 35%).

• It shows on-hand tactile beats human-video scaling on force-critical tasks. T-Rex's baseline is EgoScale (the human-video scaling law, L1) — and a 100 h tactile teleop dataset + reactive architecture nearly doubles it on delicate/deformable tasks. Concrete evidence for the pyramid's §4 point: tactile/force is the signal the human-video base can't carry, and it must be re-injected at L6.
• Variable-rate MoT is the architectural fix for tactile latency. Prior VLAs either ignore tactile or let a static encoder throttle the loop. T-Rex's fast/slow expert split (a cousin of the dual-rate designs in Review-VLA-Hybrid-Architectures) lets touch servo at 300 Hz while the VLA plans at 30 Hz — "servo on the sensor" done inside one model.
• A data-efficient recipe. 100 h, built from 22 motor primitives, argues that elementary-primitive coverage beats brute demonstration volume for contact-rich skills.

flowchart LR
  V[vision + language] --> AE[Action Expert · slow<br/>τ∈[0.4,1] · visuomotor planning · ~30 Hz]
  T[fingertip tactile · 5/hand<br/>6-axis wrench + deformation] --> TE[Tactile Expert · fast<br/>τ∈[0,0.4] · real-time tactile · 300 Hz]
  AE <-. joint attention .-> TE
  subgraph ENC[Temporal tactile VQ-VAE]
    F[16-frame force history/finger] --> Q[discrete tokens]
    NOW[current force] --> BYP[bypass — instantaneous]
    D[deformation map] --> R18[frozen ResNet-18 → 128-d]
  end
  ENC --> TE
  AE ==> OUT[action chunk]
  TE ==> OUT

• Variable-rate MoT: the two experts denoise different noise-level bands (τ), so the tactile stream refines the last, fastest part of the action while the visuomotor stream sets the plan.
• Tactile encoding: temporal VQ-VAE for force histories + bypass for instantaneous force + ResNet-18 for deformation — preserving VLA capability while adding high-rate touch.
• Hardware: Vega-1 + 2× Sharpa Wave (22 DoF each); 10 fingertip sensors total; 300 Hz control.

12 tasks (delicate force / deformable), success rate:

• Average 65% vs strongest baseline EgoScale 35% → +30 points.
• Dataset: 100 h teleoperated bimanual; 200+ objects; 22 motor primitives; synced RGB + tactile + state + action + language. Collected with Manus gloves + VIVE trackers, manufacturer IK retargeting.

Significance. T-Rex is the clearest 2026 case that reactive tactile control at sensor rate — via a variable-rate MoT + temporal tactile tokens — is what unlocks delicate/deformable dexterity that vision-and-scaling alone can't. It doubles a strong human-video baseline on exactly the tasks where force matters.

Limitations (authors').

1. Long-horizon / tight-tolerance tasks are bottlenecked by teleoperation difficulty — future work: RL or online refinement.
2. Tactile hardware constraints: sensor distortion, calibration drift across devices, no dense palm sensing.
3. Future: unified representations across heterogeneous tactile sensors; richer whole-hand tactile hardware.
4. arXiv preprint — unreplicated externally.

• Paper: arXiv 2606.17055
• Pyramid placement: L6 target-hand teleop (100 h) + tactile/force cross-cut (core) — Dexterous-Hand Data Pyramid
• Tactile line: Tactile VLA · OmniVTA · DECO · EgoTactile
• Architecture kin: VLA Hybrid Architectures (dual-rate MoT) · beaten baseline: EgoScale

← Back to Reviews · Home