aiqbench analysis
LIBERO-MAX introduces a rigorous benchmark exposing significant fragility in current Vision-Language-Action (VLA) policies when mid-execution environmental changes occur. The study quantifies how geometric shifts, viewpoint alterations, and clutter degrade task success rates by up to 25.7 percentage points across fourteen state-of-the-art models. This work is critical for advancing robust robot learning, as it isolates temporal vulnerabilities that standard static benchmarks fail to capture.
LIBERO-MAX: Do Robot Policies Adapt When the World Changes?
arXiv:2609.36518v1 Announce Type: new Abstract: Robots must often continue a task after a target moves, the viewpoint shifts, or an obstacle appears, even though their earlier observations and committed actions reflect the previous scene. Many simulation robustness benchmarks fix external conditions at reset, leaving this temporal challenge underexamined. We introduce LIBERO-MAX, a benchmark of 8,000 paired cases spanning eight types of changes to geometry, observations, appearance, clutter, and paths. Each pair compares task execution with and without a mid-task event, holding the task, initial state, policy seed, and pre-event action sequence fixed. This controlled comparison distinguishes event-associated regressions from failures already present without the change. Across fourteen current VLA, hybrid, and world-action policies, events reduce success by 11.0-25.7 percentage points. Event profiles reveal shared vulnerabilities to geometry and observation changes, while policy-family rankings interleave. Camera controls show that robustness reflects both competence under the changed conditions and the trajectory from which they are encountered; varying query cadence does not eliminate the gap. Together, the paired protocol and temporal diagnostics establish LIBERO-MAX as a reproducible testbed for diagnosing failures under mid-execution changes and measuring progress toward robot policies that remain effective as the world changes.
Title:LIBERO-MAX: Do Robot Policies Adapt When the World Changes?
Abstract:Robots must often continue a task after a target moves, the viewpoint shifts, or an obstacle appears, even though their earlier observations and committed actions reflect the previous scene. Many simulation robustness benchmarks fix external conditions at reset, leaving this temporal challenge underexamined. We introduce LIBERO-MAX, a benchmark of 8,000 paired cases spanning eight types of changes to geometry, observations, appearance, clutter, and paths. Each pair compares task execution with and without a mid-task event, holding the task, initial state, policy seed, and pre-event action sequence fixed. This controlled comparison distinguishes event-associated regressions from failures already present without the change. Across fourteen current VLA, hybrid, and world-action policies, events reduce success by 11.0-25.7 percentage points. Event profiles reveal shared vulnerabilities to geometry and observation changes, while policy-family rankings interleave. Camera controls show that robustness reflects both competence under the changed conditions and the trajectory from which they are encountered; varying query cadence does not eliminate the gap. Together, the paired protocol and temporal diagnostics establish LIBERO-MAX as a reproducible testbed for diagnosing failures under mid-execution changes and measuring progress toward robot policies that remain effective as the world changes.
42 pages, 15 figures. Project page: this https URL
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2609.36518 [cs.RO]
(or arXiv:2609.36518v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.36518 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 303 words
Related coverage
GreenWAM: A World Foundation Model for Robotics
arXiv · Oct 14, 2026
Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models
NVIDIA · Jun 15, 2026
MorphAct: Robot Policies Morphing with the Worlds They Act In
arXiv · Oct 15, 2026
Learning to Accelerate Vision-Language-Action Models through Adaptive Visual Token Caching
arXiv · Aug 27, 2026