aiqbench analysis
FingerEye advances dexterous robotics by integrating continuous vision-tactile feedback to bridge the perception gap between pre-contact approach and post-contact control. The framework's group-structured modality fusion effectively mitigates reliance on single-sensor shortcuts, yielding a 30 percentage point improvement in task success rates across diverse real-world scenarios. This work establishes a robust real-and-sim infrastructure that offers a scalable blueprint for developing more robust robotic manipulation policies.
FingerEye: Learning Dexterous Manipulation with Continuous Vision-Tactile Sensing
arXiv:2604.20689v3 Announce Type: replace Abstract: Dexterous robotic manipulation requires perception that remains informative from pre-contact approach to contact initiation and post-contact control. We introduce FingerEye, a sensing and learning framework that strengthens robotic dexterity through continuous vision-tactile feedback throughout interaction. On the sensing side, FingerEye integrates binocular RGB cameras with a compliant contact interface to support perception both before and after contact. Before contact, the fingertip cameras provide close-range visual cues and implicit stereo for precise approach and object localization. After contact, marker-tracked deformation of the compliant ring provides a proxy for contact wrench sensing. On the learning side, we build real-and-sim infrastructure for data collection and evaluation, systematically study policy-interface designs for learning with multiple FingerEye sensors, and develop FingerEye Policy, which applies group-structured modality fusion to reduce modality shortcuts and better exploit distributed fingertip feedback. Across seven contact-sensitive task settings, FingerEye improves wrist-only policy by over 30 percentage points in mean success rate in both simulation and the real world.
Abstract:Dexterous robotic manipulation requires perception that remains informative from pre-contact approach to contact initiation and post-contact control. We introduce FingerEye, a sensing and learning framework that strengthens robotic dexterity through continuous vision-tactile feedback throughout interaction. On the sensing side, FingerEye integrates binocular RGB cameras with a compliant contact interface to support perception both before and after contact. Before contact, the fingertip cameras provide close-range visual cues and implicit stereo for precise approach and object localization. After contact, marker-tracked deformation of the compliant ring provides a proxy for contact wrench sensing. On the learning side, we build real-and-sim infrastructure for data collection and evaluation, systematically study policy-interface designs for learning with multiple FingerEye sensors, and develop FingerEye Policy, which applies group-structured modality fusion to reduce modality shortcuts and better exploit distributed fingertip feedback. Across seven contact-sensitive task settings, FingerEye improves wrist-only policy by over 30 percentage points in mean success rate in both simulation and the real world.
Robotics (cs.RO)
arXiv:2604.20689 [cs.RO]
(or arXiv:2604.20689v3 [cs.RO] for this version)
Full text under the publisher's open licence · 167 words
Related coverage
FingerEye: Learning Dexterous Manipulation with Continuous Vision-Tactile Sensing
arXiv · Oct 7, 2026
Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics
Latent Space · Oct 13, 2026
Beyond Dexterity: Why Contact May Define the Next Era of Robotics
IEEE Spectrum · Jun 9, 2026
DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation
arXiv · Sep 22, 2026