analiza aiqbench
This research introduces MiTaS, a framework that fuses heterogeneous tactile sensors with varying temporal resolutions to significantly improve success rates in contact-rich robotic manipulation tasks. By leveraging a transformer-based fusion architecture, the approach achieves an 80% average success rate, substantially outperforming vision-only and standard visual-tactile baselines. The study validates the critical role of multi-resolution sensor data, demonstrating that co-training with diverse tactile inputs boosts performance even when specific high-frequency sensors are unavailable during evaluation.
Multi-Resolution Tactile Imitation Learning for Contact-Rich Robotic Manipulation
arXiv:2606.06281v1 Announce Type: new Abstract: Touch sensing is beneficial for solving a wide variety of manipulation tasks. While there exists a wide range of tactile sensors with different properties, exploiting the fusion of multiple heterogeneous tactile sensors to improve manipulation learning remains underexplored. We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions in order to solve complex contact-rich manipulation tasks. We propose a novel architecture using modality-specific convolutional stems and transformer-based fusion that effectively fuses information from an RGB camera stream, a vision-based GelSight Mini sensor and a high-frequency event-based Evetac sensor. This multi-sensor representation then conditions a flow-matching policy for solving downstream tasks. Experimental results across five contact-rich manipulation tasks demonstrate the effectiveness of multi-resolution tactile features in imitation learning. MiTaS achieves an average success rate of 80 %, while vision-only (31 %) and visual-tactile (54 %) baselines cannot solve the task reliably. Co-training a visuo-tactile model with multi-tactile data boosts performance by over 10 \% in certain tasks, without having access to the Evetac sensor during policy evaluation. A detailed sensor-reading and attention analysis reveals the importance of different sensors throughout task execution, validating our multi-resolution tactile sensing approach. Project Page: http://mitas-touch.github.io.
Abstract:Touch sensing is beneficial for solving a wide variety of manipulation tasks. While there exists a wide range of tactile sensors with different properties, exploiting the fusion of multiple heterogeneous tactile sensors to improve manipulation learning remains underexplored. We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions in order to solve complex contact-rich manipulation tasks. We propose a novel architecture using modality-specific convolutional stems and transformer-based fusion that effectively fuses information from an RGB camera stream, a vision-based GelSight Mini sensor and a high-frequency event-based Evetac sensor. This multi-sensor representation then conditions a flow-matching policy for solving downstream tasks. Experimental results across five contact-rich manipulation tasks demonstrate the effectiveness of multi-resolution tactile features in imitation learning. MiTaS achieves an average success rate of 80 %, while vision-only (31 %) and visual-tactile (54 %) baselines cannot solve the task reliably. Co-training a visuo-tactile model with multi-tactile data boosts performance by over 10 \% in certain tasks, without having access to the Evetac sensor during policy evaluation. A detailed sensor-reading and attention analysis reveals the importance of different sensors throughout task execution, validating our multi-resolution tactile sensing approach. Project Page: this http URL.
Robotics (cs.RO)
arXiv:2606.06281 [cs.RO]
(or arXiv:2606.06281v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2606.06281 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 294 cuvinte
Subiecte conexe
Temporal Visuo-Tactile Learning for Dexterous Grasp Stability
arXiv · 8 oct. 2026
A Hierarchical Approach to Imitation Learning for Manipulation Tasks Requiring Time Varying Forces
arXiv · 5 aug. 2026
Beyond Dexterity: Why Contact May Define the Next Era of Robotics
IEEE Spectrum · 9 iun. 2026
Vision-Based Tactile Intelligence for Robotics: Sensing, Learning, and Embodied Manipulation
arXiv · 18 aug. 2026