aiqbench analysis
This research investigates the geometric consistency of Vision-Language-Action policies by analyzing end-effector trajectories across 15,000 closed-loop rollouts. The findings reveal that task success alone is insufficient to guarantee physical execution agreement, as measurable residual differences persist even between successful policy pairs. These results highlight critical limitations in assuming interchangeability among different VLA architectures for robotic manipulation tasks.
Outcome-Conditioned End-Effector Geometry Across Vision-Language-Action Policies
arXiv:2609.21659v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies solve the same manipulation task through different action interfaces, but task success alone does not establish whether their physical executions agree. We study cross-policy end-effector geometry in 15,000 closed-loop LIBERO rollouts from four policies. The primary clean-condition analysis forms 3,600 configuration-matched, and therefore dependent, policy pairs. Both-success pairs have a median normalized dynamic time warping distance of 0.0120 m versus 0.0380 m when exactly one policy succeeds. This ordering holds in every task, every policy pair, and nine sampling and band-limited representations; however, the ratio varies severalfold across representations, so we report the direction rather than a fixed multiple. Both-failure pairs are more separated again but rest on thin, uneven support, so we report them as exploratory. Within successful executions, partner replacements separate more across tasks than across initial states. A matched baseline still reveals measurable, heterogeneous residual policy differences, so a low cross-policy distance does not imply interchangeability. Successful executions sit about as far from same-task demonstrations as those demonstrations sit from each other, compatible with task-associated geometry without separating training-data overlap from task constraints. A common 72-action window preserves the ordering but reduces its magnitude; endpoint and duration adjustment likewise leaves a positive mixed-outcome coefficient relative to both-success pairs, though its magnitude is specification-dependent. Under composite visual stress, policy rankings and pair composition change together.
Abstract:Vision-language-action (VLA) policies solve the same manipulation task through different action interfaces, but task success alone does not establish whether their physical executions agree. We study cross-policy end-effector geometry in 15,000 closed-loop LIBERO rollouts from four policies. The primary clean-condition analysis forms 3,600 configuration-matched, and therefore dependent, policy pairs. Both-success pairs have a median normalized dynamic time warping distance of 0.0120 m versus 0.0380 m when exactly one policy succeeds. This ordering holds in every task, every policy pair, and nine sampling and band-limited representations; however, the ratio varies severalfold across representations, so we report the direction rather than a fixed multiple. Both-failure pairs are more separated again but rest on thin, uneven support, so we report them as exploratory. Within successful executions, partner replacements separate more across tasks than across initial states. A matched baseline still reveals measurable, heterogeneous residual policy differences, so a low cross-policy distance does not imply interchangeability. Successful executions sit about as far from same-task demonstrations as those demonstrations sit from each other, compatible with task-associated geometry without separating training-data overlap from task constraints. A common 72-action window preserves the ordering but reduces its magnitude; endpoint and duration adjustment likewise leaves a positive mixed-outcome coefficient relative to both-success pairs, though its magnitude is specification-dependent. Under composite visual stress, policy rankings and pair composition change together.
8 pages, 3 figures, 7 tables, 23 references. Submitted to ICRA 2027
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2609.21659 [cs.RO]
(or arXiv:2609.21659v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.21659 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 328 words
Related coverage
From Pixels to Executable Physics: Behavior-Grounded World Models for Embodied Action Consequence Prediction
arXiv · Oct 14, 2026

How to Evaluate General-Purpose Robot Policies for Real-World Deployment
NVIDIA · Jul 12, 2026
WBAG: A Whole-Body and Attached-Geometry Safety Framework for Vision-Language-Action Manipulation
arXiv · Oct 2, 2026
Magic-W0: A Structured World-Action Foundation Model for Physical Intelligence
arXiv · Oct 1, 2026