aiqbench analysis
ResTacVLA addresses a critical bottleneck in robotics by using residual learning to prevent visual features from overwhelming sparse tactile data during contact-rich manipulation. By modeling the discrepancy between expected visual inputs and actual physical sensations, this approach effectively resolves bandwidth mismatches without requiring raw high-frequency touch streams as primary input. The method demonstrates robustness against dynamic disturbances while maintaining performance across diverse tasks through adaptive gating of sensory integration.
Feeling the Unexpected: ResTacVLA for Contact-Rich Manipulation via Residual Tactile Representation
arXiv:2607.03387v3 Announce Type: replace Abstract: Tactile perception is indispensable for contact-rich manipulation, yet integrating it into Vision-Language-Action (VLA) models often induces modality collapse, where high-bandwidth visual features overshadow sparse tactile cues. Inspired by Predictive Coding, a neural mechanism where the brain attenuates predictable inputs to prioritize surprising stimuli, we propose ResTacVLA. Rather than treating tactile data as raw input, we reformulate it as a Residual Tactile Representation capturing the discrepancy between visual priors and physical sensations. By filtering out visually predictable dynamics, this formulation transforms sparse tactile signals into dense, high-value information gain, thereby inherently resolving the bandwidth mismatch. These residuals are discretized through a Vector Quantized (VQ) bottleneck into Latent Contact Primitives that capture critical events missed by vision. Analogous to the neural surprise signal, we leverage the uncertainty of the visual prior to adaptively gate tactile integration, prioritizing residuals specifically during visually unreliable phases to explicitly prevent visual dominance. Experimental results show that ResTacVLA consistently outperforms all baselines on a diverse set of contact-rich manipulation tasks, while remaining robust to unexpected dynamic disturbances. Project page: https://awilekong.github.io/ResTacVLA/
Abstract:Tactile perception is indispensable for contact-rich manipulation, yet integrating it into Vision-Language-Action (VLA) models often induces modality collapse, where high-bandwidth visual features overshadow sparse tactile cues. Inspired by Predictive Coding, a neural mechanism where the brain attenuates predictable inputs to prioritize surprising stimuli, we propose ResTacVLA. Rather than treating tactile data as raw input, we reformulate it as a Residual Tactile Representation capturing the discrepancy between visual priors and physical sensations. By filtering out visually predictable dynamics, this formulation transforms sparse tactile signals into dense, high-value information gain, thereby inherently resolving the bandwidth mismatch. These residuals are discretized through a Vector Quantized (VQ) bottleneck into Latent Contact Primitives that capture critical events missed by vision. Analogous to the neural surprise signal, we leverage the uncertainty of the visual prior to adaptively gate tactile integration, prioritizing residuals specifically during visually unreliable phases to explicitly prevent visual dominance. Experimental results show that ResTacVLA consistently outperforms all baselines on a diverse set of contact-rich manipulation tasks, while remaining robust to unexpected dynamic disturbances. Project page: this https URL
8 pages, 6 figures, 3 tables. Accepted by IROS 2026, Project page: this https URL
Robotics (cs.RO)
arXiv:2607.03387 [cs.RO]
(or arXiv:2607.03387v3 [cs.RO] for this version)
Full text under the publisher's open licence · 200 words
Related coverage
VT-Bridge: Bridging Pretrained Foundation VLAs to VTLAs via Lightweight Residual Adaptation
arXiv · Sep 22, 2026
ViTaR: Visuo-Tactile Residual Adaptation for Foundation VLA Manipulation
arXiv · Aug 18, 2026
RATE: Risk-Aware Tactile Encoding for Contact-rich Robotic Manipulation
arXiv · Oct 5, 2026
Amazon and University of Michigan give robots a sense of touch
Amazon Science · Jul 10, 2026