analiza aiqbench
This research introduces VISTA-Policy, a machine learning framework that leverages visual deformation fields to infer contact states for robotic manipulation without requiring expensive tactile sensors. By integrating physics-aware encoding and energy aggregation denoising, the model achieves superior performance in fine-grained tasks like unscrewing caps and writing calligraphy compared to pure-vision or tactile baselines. The approach offers a cost-effective pathway toward general-purpose robots capable of handling unstructured environments with robust out-of-distribution generalization.
VISTA: Visually Inferred Spatial ConTact Attention for Contact-Rich Manipulation
arXiv:2608.25872v1 Announce Type: new Abstract: Contact-rich manipulation requires precise interaction feedback. While vision-centric imitation learning is prevalent, external visual observations provide indirect and ambiguous cues about contact states, particularly under occlusion or subtle object--gripper interactions; dedicated tactile or force sensors can provide rich contact information but introduce additional hardware complexity, calibration requirements, and deployment costs. To bridge this gap, we propose VISTA-Policy, an imitation learning paradigm that utilizes the Visual Deformation Field (VDF), a 3D displacement representation of a compliant gripper, as high-dimensional visuo-physical feedback. The framework integrates: 1) a Physics-Aware Encoding Engine for real-time VDF decoding; 2) an Energy Aggregation Denoising Mechanism to isolate true interaction signals; and 3) a Deformation-Augmented Policy Network with incremental gripper actions for precise closed-loop correction. Extensive evaluations on Cross-Scale Object Grasping, Cap Unscrewing, and Calligraphy Writing demonstrate that VISTA-Policy outperforms the strong pure-vision baseline 3D Diffusion Policy and the tactile baseline. VISTA-Policy further demonstrates substantial out-of-distribution generalization to unseen object scales and robustness against dynamic disturbances, offering a durable and cost-effective route toward general-purpose fine-grained manipulation in unstructured environments. Project videos and supplementary materials are available at: https://sites.google.com/view/vista-policy.
Abstract:Contact-rich manipulation requires precise interaction feedback. While vision-centric imitation learning is prevalent, external visual observations provide indirect and ambiguous cues about contact states, particularly under occlusion or subtle object--gripper interactions; dedicated tactile or force sensors can provide rich contact information but introduce additional hardware complexity, calibration requirements, and deployment costs. To bridge this gap, we propose VISTA-Policy, an imitation learning paradigm that utilizes the Visual Deformation Field (VDF), a 3D displacement representation of a compliant gripper, as high-dimensional visuo-physical feedback. The framework integrates: 1) a Physics-Aware Encoding Engine for real-time VDF decoding; 2) an Energy Aggregation Denoising Mechanism to isolate true interaction signals; and 3) a Deformation-Augmented Policy Network with incremental gripper actions for precise closed-loop correction. Extensive evaluations on Cross-Scale Object Grasping, Cap Unscrewing, and Calligraphy Writing demonstrate that VISTA-Policy outperforms the strong pure-vision baseline 3D Diffusion Policy and the tactile baseline. VISTA-Policy further demonstrates substantial out-of-distribution generalization to unseen object scales and robustness against dynamic disturbances, offering a durable and cost-effective route toward general-purpose fine-grained manipulation in unstructured environments. Project videos and supplementary materials are available at: this https URL.
Robotics (cs.RO)
arXiv:2608.25872 [cs.RO]
(or arXiv:2608.25872v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2608.25872 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 275 cuvinte
Subiecte conexe
PredTac: Learning Contact-Rich Manipulation with Predicted Touch
arXiv · 15 sept. 2026
PolyUMI: Accessible Visual-Tactile-Audio Data Collection for Object Inference and Manipulation
arXiv · 25 sept. 2026
Equivariant Visual-Tactile Diffusion Policy for Contact-Rich Manipulation
arXiv · 6 oct. 2026
Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
arXiv · 9 iun. 2026