analiza aiqbench
This research addresses the critical bottleneck of scarce failure data in robotics by introducing an automated framework to synthesize diverse planning and execution errors. The resulting Guardian model leverages Vision-Language Models to achieve state-of-the-art verification performance across simulated and real-world benchmarks, directly enhancing the robustness of robotic manipulation policies. By scaling high-quality failure reasoning datasets, this work establishes a new pathway for improving generalization in autonomous robot systems.
Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models
arXiv:2512.01946v4 Announce Type: replace Abstract: Robust robotic manipulation requires reliable failure detection and recovery. Although recent Vision-Language Models (VLMs) show promise in robot failure detection, their generalization is severely limited by the scarcity and narrow coverage of failure data. To address this bottleneck, we propose an automatic framework for generating diverse robotic planning and execution failures across both simulated and real-world environments. Our approach perturbs successful manipulation trajectories to synthesize failures that reflect realistic failure distributions, and leverages VLMs to produce structured step-by-step reasoning traces. This yields GuardianFail-36k, a large-scale failure reasoning dataset built upon the RLBench simulator and the BridgeDataV2 real-robot dataset. Using GuardianFail-36k, we train Guardian, a multi-view reasoning VLM for unified planning and execution verification. Guardian achieves state-of-the-art performance on three unseen real-world benchmarks: RoboFail, RoboVQA, and our newly introduced UR5-Fail. When integrated with a state-of-the-art LLM-based manipulation policy, it consistently boosts task success rates in both simulation and real-world deployment. These results demonstrate that scaling high-quality failure reasoning data is critical for improving generalization in robotic failure detection. Code, Data, and Models are available at https://www.di.ens.fr/willow/research/guardian/.
Abstract:Robust robotic manipulation requires reliable failure detection and recovery. Although recent Vision-Language Models (VLMs) show promise in robot failure detection, their generalization is severely limited by the scarcity and narrow coverage of failure data. To address this bottleneck, we propose an automatic framework for generating diverse robotic planning and execution failures across both simulated and real-world environments. Our approach perturbs successful manipulation trajectories to synthesize failures that reflect realistic failure distributions, and leverages VLMs to produce structured step-by-step reasoning traces. This yields GuardianFail-36k, a large-scale failure reasoning dataset built upon the RLBench simulator and the BridgeDataV2 real-robot dataset. Using GuardianFail-36k, we train Guardian, a multi-view reasoning VLM for unified planning and execution verification. Guardian achieves state-of-the-art performance on three unseen real-world benchmarks: RoboFail, RoboVQA, and our newly introduced UR5-Fail. When integrated with a state-of-the-art LLM-based manipulation policy, it consistently boosts task success rates in both simulation and real-world deployment. These results demonstrate that scaling high-quality failure reasoning data is critical for improving generalization in robotic failure detection. Code, Data, and Models are available at this https URL.
Code, Data, and Models available at this https URL. The paper contains 8 pages, 7 figures, 7 tables
Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2512.01946 [cs.RO]
(or arXiv:2512.01946v4 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2512.01946 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 292 cuvinte
