analiza aiqbench
This research addresses the critical bottleneck of data inefficiency in Reinforcement Learning with Verifiable Rewards (RLVR) by introducing a teacher-guided curriculum that unlocks previously unsolvable problems. By progressively withdrawing guidance from stronger models, the proposed Monotone Frontier Curriculum achieves comparable performance to standard training while using significantly less data, effectively expanding the mathematical reasoning boundary of large language models.
Unlocking the Unsolvable: Teacher-Guided Curriculum for Data-Efficient RLVR
arXiv:2609.13997v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has shown remarkable success in improving the mathematical reasoning of large language models. Yet problems beyond the model's current capability, where rollouts uniformly fail and no learning signal is produced, are structurally wasted despite marking the most informative training frontier. We show that these otherwise-inert problems can be unlocked via teacher-guided curriculum learning: partial reasoning traces from a stronger model create a graded difficulty landscape, and a backward-chaining curriculum progressively withdraws guidance until the model solves problems unaided. Training on only 128 unsolvable problems matches or exceeds GRPO trained on a full 2,000-problem corpus (~16x data efficiency) on the nine-benchmark average for both base models, while substantially expanding the reasoning boundary measured by pass@k at large k. Furthermore, we identify a distribution-shift cost that is particularly acute in the unsolvable-only regime and propose Monotone Frontier Curriculum (MFC), a method that monotonically drives training toward unguided solving, consistently outperforming existing curriculum methods.
Abstract:Reinforcement Learning with Verifiable Rewards (RLVR) has shown remarkable success in improving the mathematical reasoning of large language models. Yet problems beyond the model's current capability, where rollouts uniformly fail and no learning signal is produced, are structurally wasted despite marking the most informative training frontier. We show that these otherwise-inert problems can be unlocked via teacher-guided curriculum learning: partial reasoning traces from a stronger model create a graded difficulty landscape, and a backward-chaining curriculum progressively withdraws guidance until the model solves problems unaided. Training on only 128 unsolvable problems matches or exceeds GRPO trained on a full 2,000-problem corpus (~16x data efficiency) on the nine-benchmark average for both base models, while substantially expanding the reasoning boundary measured by pass@k at large k. Furthermore, we identify a distribution-shift cost that is particularly acute in the unsolvable-only regime and propose Monotone Frontier Curriculum (MFC), a method that monotonically drives training toward unguided solving, consistently outperforming existing curriculum methods.
20 pages, 9 figures. Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026
Computation and Language (cs.CL)
arXiv:2609.13997 [cs.CL]
(or arXiv:2609.13997v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.13997 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 267 cuvinte
Subiecte conexe
Frontier post-training recipe review with Finbarr Timbers
Interconnects · 16 iun. 2026
Granite 4.2 LLMs: How They're Built
Hugging Face · 25 aug. 2026
NVIDIA AI Releases Nemotron 3 Ultra: An Open 550B Mixture-of-Experts Hybrid Mamba-Transformer for Long-Running Agents
MarkTechPost · 4 iun. 2026
sGPO: Trading Inference FLOPs for Training Efficiency in RLVR
arXiv · 9 iun. 2026