analiza aiqbench
TacVLA introduces a contact-aware gating mechanism that integrates tactile data into Vision-Language-Action models, significantly improving robotic manipulation under visual occlusion and during physical interaction. By selectively activating tactile tokens only upon contact detection, the architecture avoids irrelevant interference while strengthening cross-modal grounding for fine-grained tasks like disassembly and in-box picking. This advancement addresses critical limitations in current VLA systems regarding robustness to environmental disturbances and lack of haptic feedback.
TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation
arXiv:2603.12665v3 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have demonstrated significant advantages in robotic manipulation. However, their reliance on vision and language often leads to suboptimal performance in tasks involving visual occlusion, fine-grained manipulation, and physical contact. To address these challenges, we propose TacVLA, a fine-tuned VLA model by incorporating tactile modalities into the transformer-based policy to enhance fine-grained manipulation capabilities. Specifically, we introduce a contact-aware gating mechanism that selectively activates tactile tokens only when contact is detected, enabling adaptive multimodal fusion while avoiding irrelevant tactile interference. The fused visual, language, and tactile tokens are jointly processed within the transformer architecture to strengthen cross-modal grounding during contact-rich interaction. Extensive experiments on constraint-locked disassembly, in-box picking and robustness evaluations demonstrate that TacVLA outperforms baselines, %including existing VLA models and diffusion policies, improving the performance by averaging 20\% success rate in disassembly and 60\% in in-box picking, achieving a 2.1$\times$ improvement under visual occlusion, and showing recovery behavior under human disturbance. Videos are available at https://sites.google.com/view/tacvla.
Abstract:Vision-Language-Action (VLA) models have demonstrated significant advantages in robotic manipulation. However, their reliance on vision and language often leads to suboptimal performance in tasks involving visual occlusion, fine-grained manipulation, and physical contact. To address these challenges, we propose TacVLA, a fine-tuned VLA model by incorporating tactile modalities into the transformer-based policy to enhance fine-grained manipulation capabilities. Specifically, we introduce a contact-aware gating mechanism that selectively activates tactile tokens only when contact is detected, enabling adaptive multimodal fusion while avoiding irrelevant tactile interference. The fused visual, language, and tactile tokens are jointly processed within the transformer architecture to strengthen cross-modal grounding during contact-rich interaction. Extensive experiments on constraint-locked disassembly, in-box picking and robustness evaluations demonstrate that TacVLA outperforms baselines, %including existing VLA models and diffusion policies, improving the performance by averaging 20\% success rate in disassembly and 60\% in in-box picking, achieving a 2.1$\times$ improvement under visual occlusion, and showing recovery behavior under human disturbance. Videos are available at this https URL.
Robotics (cs.RO)
arXiv:2603.12665 [cs.RO]
(or arXiv:2603.12665v3 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2603.12665 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 252 cuvinte
Subiecte conexe
DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination
arXiv · 9 sept. 2026
ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation
arXiv · 21 sept. 2026
Digital States 2026: AI Moves From Experiment to Infrastructure
govtech.com · 24 sept. 2026
HapticVLA: Contact-Rich Manipulation via Vision-Language-Action Model without Inference-Time Tactile Sensing
arXiv · 4 aug. 2026