aiqbench analysis
AcceRL addresses critical bottlenecks in training Vision-Language-Action models by introducing a distributed asynchronous framework that physically isolates environment rollouts from inference and gradient updates. This architecture eliminates synchronization barriers to achieve significant hardware utilization gains, delivering up to 200x better sample efficiency when leveraging pre-trained world models for embodied AI tasks. The approach represents a substantial methodological advance in scaling reinforcement learning without relying on synchronous system constraints that limit throughput.
AcceRL: A Distributed Asynchronous Reinforcement Learning and World Model Framework for Vision-Language-Action Models
arXiv:2603.18464v3 Announce Type: replace Abstract: Reinforcement learning (RL) for large-scale Vision-Language-Action (VLA) models is severely bottlenecked by synchronization barriers and the high cost of environment data acquisition. To overcome these challenges, we propose AcceRL, a distributed asynchronous RL framework that physically isolates environment rollouts, model inference, and gradient updates. By eliminating the cascading long-tail idle bubbles inherent in synchronous systems, AcceRL maximizes hardware utilization and ensures scalable throughput. Furthermore, AcceRL features a modular design that supports the integration of diverse, plug-and-play world models into its distributed pipeline. Extensive experiments demonstrate that the base framework achieves highly competitive performance across all four LIBERO~\cite{liu2023libero} task suites. Systematically, the asynchronous architecture delivers a $2.4\times$ throughput speedup over leading synchronous baselines. Algorithmically, by leveraging a world model pre-trained on 1,000 offline trajectories, AcceRL achieves up to a $200\times$ improvement in online sample efficiency on LIBERO-Spatial, establishing a robust framework that is both sample-efficient and time-efficient for embodied AI. Code is included in the supplementary material. Code is available at https://github.com/distanceLu/AcceRL.
Abstract:Reinforcement learning (RL) for large-scale Vision-Language-Action (VLA) models is severely bottlenecked by synchronization barriers and the high cost of environment data acquisition. To overcome these challenges, we propose AcceRL, a distributed asynchronous RL framework that physically isolates environment rollouts, model inference, and gradient updates. By eliminating the cascading long-tail idle bubbles inherent in synchronous systems, AcceRL maximizes hardware utilization and ensures scalable throughput. Furthermore, AcceRL features a modular design that supports the integration of diverse, plug-and-play world models into its distributed pipeline. Extensive experiments demonstrate that the base framework achieves highly competitive performance across all four LIBERO~\cite{liu2023libero} task suites. Systematically, the asynchronous architecture delivers a $2.4\times$ throughput speedup over leading synchronous baselines. Algorithmically, by leveraging a world model pre-trained on 1,000 offline trajectories, AcceRL achieves up to a $200\times$ improvement in online sample efficiency on LIBERO-Spatial, establishing a robust framework that is both sample-efficient and time-efficient for embodied AI. Code is included in the supplementary material. Code is available at this https URL.
Machine Learning (cs.LG)
arXiv:2603.18464 [cs.LG]
(or arXiv:2603.18464v3 [cs.LG] for this version)
Full text under the publisher's open licence · 174 words
Related coverage
Reinforcement Learning for Real-Time Vision-Language-Action Policies
arXiv · Sep 17, 2026
SmoothRL: Online Reinforcement Learning During Asynchronous Execution
arXiv · Sep 2, 2026
Introducing North Mini Code: Cohere’s First Model For Developers
Hugging Face · Jun 9, 2026
Out-of-the-Box ROLL Support on AMD GPUs: Accelerating Reinforcement Learning at Scale
AMD · Jun 1, 2026