Touch, not more video, is making dexterous robots work
Touch data, not just more video, is what's unlocking dexterous robots
Four separate teams show tactile feedback roughly doubling success on fiddly tasks — but nobody agrees yet on how much data or which architecture wins
Tactile sensing is emerging as the missing ingredient for fine manipulation that vision-language-action models can't do — but the field is still fragmented across incompatible sensors, unproven data-scale assumptions, and competing architectures for using touch fast enough to matter.
Why touch was missing
VLA models learned everyday tasks from huge vision/language pretraining plus light robot fine-tuning, but they ignore touch entirely. That's fine for coarse tasks, but fails at things like plugging in a USB cable or turning a key — jobs humans could do blindfolded using force and slip feedback. The blocker wasn't ambition but data: tactile datasets are tiny compared to internet-scale vision/text corpora, and touch signals don't look like images.
Four teams, four approaches
- T-Rex: fast tactile expert + slow action expert
100 hrs of tactile demos, 200+ objects; tactile submodel runs 4x faster than the action model for real-time grip correction
- Cross-sensor aggregation
3,000+ hours pooled across 21 sensor types, mapped onto a shared human-hand template to generalize across unseen hardware
- Scale + prediction
30,000+ hours of synced visual-tactile data; model predicts expected touch to guide and assess actions
- Vision-to-touch inference
2,700 demos teaching a model to infer pressure from images alone, aiming to add synthetic touch to existing vision datasets
- Surprise-gated fusion
Predicts touch from vision, amplifies only the surprising mismatches so sparse tactile signals aren't drowned out by dense pixels
Success rate with vs. without tactile input
Two independent teams report roughly the same ~2x jump from adding tactile feedback, on different task sets.
How much data is enough? Nobody agrees
single robot hardware only
21 sensor types, many robots
proprietary sensor, arms + handheld gripper
real-world, varied settings — unverified projection
Where the field disagrees
Scale is the answer
- Lu (NeoteAI): bigger data "clear evidence" of gains, guesses ~100k hrs could unlock new capabilities
- 80-institution collaboration led by Yuan forming to standardize and expand data collection
Data alone isn't enough
- Yuan: touch so far mostly makes robots more efficient at tasks already achievable, not new ones
- Cheng: "data is good, but how to use them correctly is another issue" — sparse touch gets ignored by models unless architecture forces attention to it
The hardware fragmentation problem
Darrell's own caveat is telling: T-Rex's gains come from a single robot/sensor setup, and tactile sensors vary wildly in physics (resistance-based vs. deforming gel pads), making most tactile research non-transferable between labs. Tsinghua's hand-template mapping and the emerging 80-institution standardization effort are direct responses to this — echoing how Open X-Embodiment unlocked cross-robot generalization in vision-based control.
What to watch
- 01
IROS 2026 (this month): Long Cheng's CAS presentation on surprise-gated vision-touch fusion, with its 62.8% vs 28.2% result
- 02
Progress of the 80-institution collaboration led by Chengbo Yuan on a standardized tactile data protocol
- 03
Whether NeoteAI or others push toward Lu's ~100,000-hour target and report qualitatively new (not just more efficient) capabilities