analiza aiqbench
This research demonstrates that adaptive data selection via autocurriculum can provably reduce the training costs for chain-of-thought reasoning models, exponentially lowering the need for demonstration data in supervised fine-tuning and decoupling reinforcement learning costs from reference model quality. By leveraging classical boosting techniques to focus supervision on difficult prompts, the study offers a fundamental algorithmic approach to mitigating the high compute and data expenses currently inherent in training advanced reasoning systems. These findings suggest that significant efficiency gains are achievable through better curriculum design rather than solely relying on increased hardware or synthetic data generation.
Learning to Reason with Curriculum I: Provable Benefits of Autocurriculum
arXiv:2603.18325v2 Announce Type: replace Abstract: Chain-of-thought reasoning, where language models expend additional computation by producing thinking tokens prior to final responses, has driven significant advances in model capabilities. However, training these reasoning models is extremely costly in terms of both data and compute, as it involves collecting long traces of reasoning behavior from humans or synthetic generators and further post-training the model via reinforcement learning. Are these costs fundamental, or can they be reduced through better algorithmic design? We show that autocurriculum, where the model uses its own performance to decide which problems to focus training on, provably improves upon standard training recipes for both supervised fine-tuning (SFT) and reinforcement learning (RL). For SFT, we show that autocurriculum requires exponentially fewer reasoning demonstrations than non-adaptive fine-tuning, by focusing teacher supervision on prompts where the current model struggles. For RL fine-tuning, autocurriculum decouples the computational cost from the quality of the reference model, reducing the latter to a burn-in cost that is nearly independent of the target accuracy. These improvements arise purely from adaptive data selection, drawing on classical techniques from boosting and learning from counterexamples, and requiring no assumption on the distribution or difficulty of prompts.
Abstract:Chain-of-thought reasoning, where language models expend additional computation by producing thinking tokens prior to final responses, has driven significant advances in model capabilities. However, training these reasoning models is extremely costly in terms of both data and compute, as it involves collecting long traces of reasoning behavior from humans or synthetic generators and further post-training the model via reinforcement learning. Are these costs fundamental, or can they be reduced through better algorithmic design? We show that autocurriculum, where the model uses its own performance to decide which problems to focus training on, provably improves upon standard training recipes for both supervised fine-tuning (SFT) and reinforcement learning (RL). For SFT, we show that autocurriculum requires exponentially fewer reasoning demonstrations than non-adaptive fine-tuning, by focusing teacher supervision on prompts where the current model struggles. For RL fine-tuning, autocurriculum decouples the computational cost from the quality of the reference model, reducing the latter to a burn-in cost that is nearly independent of the target accuracy. These improvements arise purely from adaptive data selection, drawing on classical techniques from boosting and learning from counterexamples, and requiring no assumption on the distribution or difficulty of prompts.
Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2603.18325 [cs.LG]
(or arXiv:2603.18325v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2603.18325 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 286 cuvinte
Subiecte conexe
Models Can't Remember Their Training. Neither Can You.
LessWrong · 19 iul. 2026
Self-Evolve With a Reference:Anchored Training of Tool-Integrated Agents
arXiv · 8 oct. 2026
For Love of the Lightcone, Don't Partisanize AI Safety
LessWrong · 17 sept. 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · 25 sept. 2026