LightTact: A Visual-Tactile Fingertip Sensor for Deformation-Independent Contact Sensing

Venue: RSS 2026 (Sydney, Jul 13–17) · Session: Robot & Sensor Design · paper #193 Authors: Changyi Lin, Boda Huo, Mingyang Yu, Emily Ruppel, Bingqing Chen, Jonathan Francis, Ding Zhao arXiv: 2512.20591 · program page

Summary compiled from the arXiv paper (v3); all numbers quoted from the paper. Trend context: RSS 2026 survey.

Figure 1: left, raw images and contact segmentations for a thin film, mango juice, a strawberry, and dice — non-contact pixels stay near-black while contact pixels show the object's natural appearance; right, fingertip-sized LightTact sensors integrated into the Amazing Hand dexterous hand, sensing a strawberry and a hanging thin film.

Problem

Most vision-based tactile sensors infer contact from macroscopic deformation of a soft surface, so near-zero-force interactions — liquids, semi-liquids, ultra-soft or thin materials — go undetected; frustrated-TIR alternatives leak ambient light and only work with monochromatic objects or in dark rooms. The goal is robust contact perception that needs no deformation and no minimum force.

Method

LightTact enforces a bijective contact-visibility relationship: a pixel is imaged if and only if it is in true physical contact. An ambient-blocking optical configuration — side-view imaging with dedicated camera/LED/transparent-medium geometry exploiting refraction, specular reflection, and total internal reflection (the layout enforces θ_tv > 2θ_c) — suppresses external light and internal illumination at non-contact regions, transmitting only contact-scattered light. The co-designed fingertip measures 12 × 18 × 34.5 mm, costs under $20 beyond the RGB camera, uses fixed 20 ms exposure, calibrates pixel-to-surface mapping with a 5×5 bump tool (3 mm spacing), and segments contact with a simple intensity-threshold algorithm; hardware, software, and a fabrication tutorial are open-sourced.

Results

Non-contact pixels stay near-black: mean gray value 1.0 at the default 430 Lux internal LED in the dark, and below 3 under ambient lighting up to 2010 Lux (6.90 at an extreme 3520 Lux) while segmentation stays robust. It cleanly segments contacts from green juice, milk, toothpaste, cotton, sponge, tofu, noodle, beef, and finger/palm prints placed with minimal or no deformation — a regime where 9DTact, GelSight, and DelTact give no reliable detection — and detects a 0.1 mm, 0.05 g thin film hanging under zero applied force. Robot demos include water spreading (PD control holding ~50% surface contact), facial-cream dipping with reliable detection during lifting, and delicate thin-film handover with two sensors; the spatially aligned visual-tactile images are directly interpretable by VLMs, e.g., reading resistor color bands without fine-tuning.

Significance

A deformation-free contact modality broadens tactile sensing to the liquid/ultra-soft regime that gel-type sensors structurally miss, and its VLM-readable raw images plug tactile input straight into foundation-model pipelines — see Review-Tactile-VLA.

← Back to RSS 2026 survey · RSS-2026-Papers · Home