ICML 2026 EgoTactile - Heungwoo/research GitHub Wiki

EgoTactile — learning full-hand grasp pressure from egocentric video

Venue: ICML 2026 (Poster) Category: Tactile

Problem

Estimating full-hand grasp pressure from egocentric video is "critical for immersive VR and robotic manipulation, yet dense tactile sensing often relies on intrusive hardware." Instrumented gloves and pressure sensors are intrusive, alter natural contact, and do not scale to everyday objects, leaving a gap for vision-only pressure estimation that works in natural, bare-hand scenarios.

Method

EgoTactile contributes both a dataset and a model.

Dataset. EgoTactile is a comprehensive dataset pairing egocentric video with full-hand pressure measurements for everyday objects, including a bare-hand transfer subset that captures natural, uninstrumented grasping scenarios.
Discriminative baseline. The authors first establish EgoPressureFormer, a discriminative baseline for the pressure-estimation task.
EgoPressureDiff. The main method is "a conditional diffusion framework that adapts a large-scale pre-trained video diffusion backbone." Reusing a pretrained video diffusion model injects world-knowledge priors about objects and contact.
Physically-Informed Feature Rectification. A dedicated layer enforces semantic/physical constraints, letting the model infer plausible contact patterns while resolving visual–physical ambiguities (e.g., occluded or visually under-determined contacts).

Together these let the model produce dense, plausible full-hand pressure maps from video alone, without intrusive tactile hardware at inference time.

flowchart LR
    V[Egocentric video] --> B[Pre-trained video diffusion backbone]
    B --> R[Physically-Informed Feature Rectification]
    R --> D[Conditional diffusion decoder]
    D --> P[Full-hand grasp pressure map]

Results

The paper reports that testing "demonstrates strong performance on the benchmark with effective generalization to real-world settings," with EgoPressureDiff improving over the EgoPressureFormer discriminative baseline. No specific numeric metrics are given in the available abstract.

Significance

EgoTactile pushes tactile/pressure sensing toward a vision-only, hardware-free regime by repurposing large pre-trained video diffusion models as physical priors. A bare-hand pressure benchmark plus a diffusion estimator is directly useful for immersive VR interaction and for supervising contact-rich robotic manipulation without instrumenting either the hand or the objects.