aiqbench analysis
RLVE-Gym demonstrates that procedurally generating verifiable environments and dynamically adjusting difficulty distributions significantly outperforms static RL training, yielding a 3.37% improvement in reasoning benchmarks with far less compute. This approach effectively solves the vanishing learning signal problem inherent in static data distributions by ensuring tasks remain optimally challenging as the model evolves. The release of this 400-environment suite provides a scalable framework for enhancing generalizable reasoning capabilities in language models.
RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments
arXiv:2511.07317v2 Announce Type: replace-cross Abstract: We introduce Reinforcement Learning (RL) with Adaptive Verifiable Environments (RLVE), an approach using verifiable environments that procedurally generate problems and provide algorithmically verifiable rewards, to scale up RL for language models (LMs). RLVE enables each verifiable environment to dynamically adapt its problem difficulty distribution to the policy model's capabilities as training progresses. In contrast, static data distributions often lead to vanishing learning signals when problems are either too easy or too hard for the policy. To implement RLVE, we create RLVE-Gym, a large-scale suite of 400 verifiable environments carefully developed through manual environment engineering. Using RLVE-Gym, we show that environment scaling, i.e., expanding the collection of training environments, consistently improves generalizable reasoning capabilities. RLVE with joint training across all 400 environments in RLVE-Gym yields a 3.37% absolute average improvement across six reasoning benchmarks, starting from one of the strongest 1.5B reasoning LMs. By comparison, continuing this LM's original RL training yields only a 0.49% average absolute gain despite using over 3x more compute. We release our code publicly.
Abstract:We introduce Reinforcement Learning (RL) with Adaptive Verifiable Environments (RLVE), an approach using verifiable environments that procedurally generate problems and provide algorithmically verifiable rewards, to scale up RL for language models (LMs). RLVE enables each verifiable environment to dynamically adapt its problem difficulty distribution to the policy model's capabilities as training progresses. In contrast, static data distributions often lead to vanishing learning signals when problems are either too easy or too hard for the policy. To implement RLVE, we create RLVE-Gym, a large-scale suite of 400 verifiable environments carefully developed through manual environment engineering. Using RLVE-Gym, we show that environment scaling, i.e., expanding the collection of training environments, consistently improves generalizable reasoning capabilities. RLVE with joint training across all 400 environments in RLVE-Gym yields a 3.37% absolute average improvement across six reasoning benchmarks, starting from one of the strongest 1.5B reasoning LMs. By comparison, continuing this LM's original RL training yields only a 0.49% average absolute gain despite using over 3x more compute. We release our code publicly.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2511.07317 [cs.CL]
(or arXiv:2511.07317v2 [cs.CL] for this version)
Full text under the publisher's open licence · 183 words
Related coverage
Simulated Users & Sad AIs
LessWrong · Jul 27, 2026
Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk
Latent Space · Oct 8, 2026
From Ticks to Flows: Dynamics of Neural Reinforcement Learning in Continuous Environments
arXiv · Jun 5, 2026
Learning to See via Epiretinal Implant Stimulation in silico with Model-Based Deep Reinforcement Learning
arXiv · Jun 3, 2026