aiqbench analysis
This paper applies transformer-based self-supervised pretraining, using masked reconstruction and cross-modal alignment, to tactile sensors that differ in modality and hardware. Its main claim is that a shared representation can transfer to new tasks and unseen sensors, which would reduce the need to retrain for each sensor type. Readers should weigh the results against the 1.6M-frame dataset and the reported manipulation tasks, and note that the paper is a CoRL 2026 submission whose transfer claims still need independent replication.
Heterogeneous Tactile Transformer
arXiv:2606.29948v2 Announce Type: replace Abstract: Tactile sensors are inherently heterogeneous: a model trained on one sensor cannot be directly used on another, which limits learning contact-rich manipulation policies from diverse tactile data at scale. To bridge this gap, we propose the Heterogeneous Tactile Transformer (HTT), a framework that learns shared tactile representations across heterogeneous sensors. HTT consists of sensor-specific encoders and a shared transformer trunk, and is pretrained with per-modality masked reconstruction together with cross-modal alignment between paired sensors. Pretraining uses our novel Heterogeneous Paired Tactile (HPT) dataset, containing 1.6M synchronized paired frames across four vision- and array-based tactile sensors. Across distinct tactile perception and real-world manipulation tasks, HTT is shown to learn transferable representations that adapt to new tasks and previously unseen sensors. Dataset, code, and model checkpoints are released at https://jxbi1010.github.io/htt-gh-page/.
Abstract:Tactile sensors are inherently heterogeneous: a model trained on one sensor cannot be directly used on another, which limits learning contact-rich manipulation policies from diverse tactile data at scale. To bridge this gap, we propose the Heterogeneous Tactile Transformer (HTT), a framework that learns shared tactile representations across heterogeneous sensors. HTT consists of sensor-specific encoders and a shared transformer trunk, and is pretrained with per-modality masked reconstruction together with cross-modal alignment between paired sensors. Pretraining uses our novel Heterogeneous Paired Tactile (HPT) dataset, containing 1.6M synchronized paired frames across four vision- and array-based tactile sensors. Across distinct tactile perception and real-world manipulation tasks, HTT is shown to learn transferable representations that adapt to new tasks and previously unseen sensors. Dataset, code, and model checkpoints are released at this https URL.
15 pages, 5 figures. Paper has been accepted to CoRL 2026
Robotics (cs.RO)
arXiv:2606.29948 [cs.RO]
(or arXiv:2606.29948v2 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2606.29948 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 232 words
Related coverage
TACROSS: An Efficient and Low-Cost Scalable Human Touch System Across Heterogeneous Tactile Sensors for Dexterous Robot Learning
arXiv · Oct 9, 2026
FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation
arXiv · Oct 14, 2026
FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation
arXiv · Jun 12, 2026
OmniVTLA: Vision-Tactile-Language-Action Models with Semantic-Aligned Tactile Sensing
arXiv · Jun 16, 2026