aiqbench analysis
This article details the first REAL-I Challenge at ICRA 2026, analyzing how Vision-Language-Action models can optimize learning from fixed demonstration budgets in real-world robotic settings. Key findings emphasize the necessity of adapting pretrained policies to specific deployment environments and managing data curation strategies for dual-arm humanoid platforms. The study highlights critical gaps between offline action-prediction metrics and actual closed-loop success rates in embodied AI systems.
How to Better Train VLAs: Lessons Learned From the REAL-I Challenge at ICRA 2026
arXiv:2609.13679v1 Announce Type: new Abstract: How can robot policies learn more effectively from a fixed demonstration budget? The first Real-world Embodied AI Learning (REAL-I) Challenge at ICRA 2026 examined this question through simulation, real-robot evaluation, and an on-site final on a shared dual-arm humanoid platform. We describe the challenge tasks, data and deployment interfaces, and competition results, then compare the approaches contributed by NUS-CLEAR, RCL-Lab, and Deeptouch.ai. Their systems combined pretrained vision-language-action models and task-specific imitation policies with different strategies for data curation, staged adaptation, checkpoint selection, and action-space design. The team reports highlight the importance of adapting to the deployment environment while retaining prior capabilities, treating demonstration quality at an appropriate temporal scale, and suppressing errors in inactive robot components. They also expose the limitations of offline action-prediction metrics for forecasting closed-loop success. These observations motivate a view of fixed-data robot learning that integrates data, adaptation, evaluation, and deployment.
Abstract:How can robot policies learn more effectively from a fixed demonstration budget? The first Real-world Embodied AI Learning (REAL-I) Challenge at ICRA 2026 examined this question through simulation, real-robot evaluation, and an on-site final on a shared dual-arm humanoid platform. We describe the challenge tasks, data and deployment interfaces, and competition results, then compare the approaches contributed by NUS-CLEAR, RCL-Lab, and this http URL. Their systems combined pretrained vision-language-action models and task-specific imitation policies with different strategies for data curation, staged adaptation, checkpoint selection, and action-space design. The team reports highlight the importance of adapting to the deployment environment while retaining prior capabilities, treating demonstration quality at an appropriate temporal scale, and suppressing errors in inactive robot components. They also expose the limitations of offline action-prediction metrics for forecasting closed-loop success. These observations motivate a view of fixed-data robot learning that integrates data, adaptation, evaluation, and deployment.
Robotics (cs.RO)
arXiv:2609.13679 [cs.RO]
(or arXiv:2609.13679v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.13679 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 240 words
Related coverage
Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics
Latent Space · Oct 13, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026

The Machines that Make the Machines
NVIDIA · Oct 7, 2026

Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026