aiqbench analysis
This research introduces Learning While Deploying (LWD), a framework that enables continual post-training of generalist Vision-Language-Action policies by leveraging real-world fleet data and human interventions to address distribution shifts. By combining Distributional Implicit Value Learning with Q-learning via Adjoint Matching, the system successfully improves policy robustness across a fleet of 16 dual-arm robots, achieving a 95% average success rate on long-horizon tasks. The approach demonstrates that closed-loop learning from heterogeneous, sparse-reward environments is critical for scaling robot capabilities beyond the limits of static offline datasets.
Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies
arXiv:2605.00416v2 Announce Type: replace Abstract: Generalist robot policies increasingly benefit from large-scale pretraining, but offline data alone is insufficient for robust real-world deployment. Deployed robots encounter distribution shifts, long-tail failures, task variations, and human correction opportunities that fixed demonstration datasets cannot fully capture. We present Learning While Deploying (LWD), a fleet-scale offline-to-online reinforcement learning framework for continual post-training of generalist Vision-Language-Action (VLA) policies. Starting from a pretrained VLA policy, LWD closes the loop between deployment, shared physical experience, policy improvement, and redeployment by using autonomous rollouts and human interventions collected across a robot fleet. To stabilize learning from heterogeneous, sparse-reward fleet data, LWD combines Distributional Implicit Value Learning (DIVL) for robust value estimation with Q-learning via Adjoint Matching (QAM) for policy extraction in flow-based VLA action generators. We validate LWD on a fleet of 16 dual-arm robots across eight real-world manipulation tasks, including semantic grocery restocking and 3--5 minute long-horizon tasks. A single generalist policy improves as fleet experience accumulates, reaching an average success rate of 95%, with the largest gains on long-horizon tasks.
Abstract:Generalist robot policies increasingly benefit from large-scale pretraining, but offline data alone is insufficient for robust real-world deployment. Deployed robots encounter distribution shifts, long-tail failures, task variations, and human correction opportunities that fixed demonstration datasets cannot fully capture. We present Learning While Deploying (LWD), a fleet-scale offline-to-online reinforcement learning framework for continual post-training of generalist Vision-Language-Action (VLA) policies. Starting from a pretrained VLA policy, LWD closes the loop between deployment, shared physical experience, policy improvement, and redeployment by using autonomous rollouts and human interventions collected across a robot fleet. To stabilize learning from heterogeneous, sparse-reward fleet data, LWD combines Distributional Implicit Value Learning (DIVL) for robust value estimation with Q-learning via Adjoint Matching (QAM) for policy extraction in flow-based VLA action generators. We validate LWD on a fleet of 16 dual-arm robots across eight real-world manipulation tasks, including semantic grocery restocking and 3--5 minute long-horizon tasks. A single generalist policy improves as fleet experience accumulates, reaching an average success rate of 95%, with the largest gains on long-horizon tasks.
Robotics (cs.RO)
arXiv:2605.00416 [cs.RO]
(or arXiv:2605.00416v2 [cs.RO] for this version)
Full text under the publisher's open licence · 179 words
Related coverage
Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies
arXiv · Aug 25, 2026
2026 BAIR Graduate Showcase
BAIR Blog · Jul 1, 2026
Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics
Latent Space · Oct 13, 2026
The Machines that Make the Machines
NVIDIA · Oct 7, 2026