analiza aiqbench
This research introduces SCoRe, a novel distillation framework that narrows the performance gap between small and large LLM agents by leveraging student-generated trajectories rather than simple imitation. By correcting only the earliest errors in training data and utilizing short-horizon reinforcement learning, the method enables smaller models to solve problems through unconstrained exploration while maintaining stability across 12 challenging benchmarks. This approach offers a significant pathway for deploying efficient agentic systems without relying on massive parameter counts.
Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMs
arXiv:2509.14257v3 Announce Type: replace Abstract: Large Language Model agents achieve strong performance on multi-step reasoning and tool-use tasks, but their impressive capabilities typically rely on extremely large backbones. Existing distillation approaches train smaller students to imitate full teacher trajectories, yet reasoning and knowledge gaps between the teacher and student can cause compounding errors. We propose SCoRe, a student-centered framework in which the student generates training trajectories and the teacher corrects only the earliest error, producing training data matched to the student's abilities and exposing specific weaknesses. The student is first fine-tuned on corrected trajectories. Subsequently, short-horizon reinforcement learning starts from the verified prefix preceding the earliest error, with target rewards assigned at that step. This design enables the student to solve problems through unconstrained RL exploration rather than teacher imitation, while the short-horizon setup improves training stability. On 12 challenging benchmarks, a 7B-parameter student distilled with SCoRe closes the agentic performance gap with a 72B-parameter teacher.
Abstract:Large Language Model agents achieve strong performance on multi-step reasoning and tool-use tasks, but their impressive capabilities typically rely on extremely large backbones. Existing distillation approaches train smaller students to imitate full teacher trajectories, yet reasoning and knowledge gaps between the teacher and student can cause compounding errors. We propose SCoRe, a student-centered framework in which the student generates training trajectories and the teacher corrects only the earliest error, producing training data matched to the student's abilities and exposing specific weaknesses. The student is first fine-tuned on corrected trajectories. Subsequently, short-horizon reinforcement learning starts from the verified prefix preceding the earliest error, with target rewards assigned at that step. This design enables the student to solve problems through unconstrained RL exploration rather than teacher imitation, while the short-horizon setup improves training stability. On 12 challenging benchmarks, a 7B-parameter student distilled with SCoRe closes the agentic performance gap with a 72B-parameter teacher.
Accepted to ICML 2026. The title has been changed from "From Correction to Mastery: Reinforced Distillation of Large Language Model Agents" to "Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMs"; the camera-ready version has been uploaded
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2509.14257 [cs.CL]
(or arXiv:2509.14257v3 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 205 cuvinte
Subiecte conexe
Opaque recurrence, and other AI terms that you should probably know
TechCrunch · 7 sept. 2026
Georgia Tech AI Safety Initiative Retrospective 2025-2026
LessWrong · 24 iul. 2026
Sen. Adam Schiff on AI regulation, free speech, and impeaching Trump one more time
The Verge · 5 oct. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026