analiza aiqbench
This paper introduces MiTaS, a framework leveraging heterogeneous tactile sensors and transformer-based fusion to enhance imitation learning for contact-rich robotic manipulation tasks. By integrating RGB camera data with vision-based and event-based tactile inputs, the approach achieves an 80% success rate across five complex tasks, significantly outperforming vision-only or standard visuotactile baselines. The work advances the field of robotics by demonstrating how multi-resolution sensory fusion can solve previously unreliable manipulation problems through improved policy conditioning.
Multi-Resolution Tactile Imitation Learning for Contact-Rich Robotic Manipulation
arXiv:2606.06281v2 Announce Type: replace Abstract: Touch sensing is beneficial for solving a wide variety of manipulation tasks. While there exists a wide range of tactile sensors with different properties, exploiting the fusion of multiple heterogeneous tactile sensors to improve manipulation learning remains underexplored. We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions in order to solve complex contact-rich manipulation tasks. We propose an effective architecture using modality-specific convolutional stems and transformer-based fusion that effectively integrates information from an RGB camera stream, a vision-based GelSight Mini sensor and a high-frequency event-based Evetac sensor. This multisensory representation then conditions a flow-matching policy for solving downstream tasks. Our systematic study across five contact-rich manipulation tasks demonstrates the effectiveness of multi-resolution tactile features in imitation learning wherein MiTaS achieves an average success rate of 80 \%, while vision-only (35.5\,\%) and visuotactile (55.5\,\%, 63\,\%) baselines cannot solve the task reliably. Project Page: http://mitas-touch.github.io.
Abstract:Touch sensing is beneficial for solving a wide variety of manipulation tasks. While there exists a wide range of tactile sensors with different properties, exploiting the fusion of multiple heterogeneous tactile sensors to improve manipulation learning remains underexplored. We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions in order to solve complex contact-rich manipulation tasks. We propose an effective architecture using modality-specific convolutional stems and transformer-based fusion that effectively integrates information from an RGB camera stream, a vision-based GelSight Mini sensor and a high-frequency event-based Evetac sensor. This multisensory representation then conditions a flow-matching policy for solving downstream tasks. Our systematic study across five contact-rich manipulation tasks demonstrates the effectiveness of multi-resolution tactile features in imitation learning wherein MiTaS achieves an average success rate of 80 \%, while vision-only (35.5\,\%) and visuotactile (55.5\,\%, 63\,\%) baselines cannot solve the task reliably. Project Page: this http URL.
Robotics (cs.RO)
arXiv:2606.06281 [cs.RO]
(or arXiv:2606.06281v2 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2606.06281 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 246 cuvinte