aiqbench analysis
This research introduces DualOPSD, an adaptive framework for on-policy self-distillation that dynamically aligns the privileged teacher with the evolving student distribution during training. By eliminating the need for external rollouts to update supervision, the method significantly boosts mathematical reasoning performance on benchmarks like AIME and HMMT across various model scales. The findings highlight a critical dependency on model size for accuracy gains while demonstrating consistent reductions in output truncation.
DualOPSD: Adaptive Privileged Teachers for On-Policy Self-Distillation
arXiv:2608.26019v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) uses a privileged copy of the student model to provide dense supervision without an external teacher. OPSD keeps this privileged teacher fixed, even though the student distribution and output style change during training. We propose DualOPSD, an asymmetric alternating framework that adapts both policies. The student first learns from the privileged teacher. The teacher then moves toward the updated student distribution on the same student trajectory. This update makes later supervision responsive to the learner and does not require another rollout. On Qwen3-8B in non-thinking mode, DualOPSD improves avg@12 over OPSD by 23.61, 13.89, and 10.00 points on AIME 2024, AIME 2025, and HMMT 2025. Results at 1.7B and 4B show that the accuracy gain depends on model scale. Across all three scales, DualOPSD reduces truncation. The 4B diagnostic also shows lower KL in both directions between the teacher and student.
Abstract:On-policy self-distillation (OPSD) uses a privileged copy of the student model to provide dense supervision without an external teacher. OPSD keeps this privileged teacher fixed, even though the student distribution and output style change during training. We propose DualOPSD, an asymmetric alternating framework that adapts both policies. The student first learns from the privileged teacher. The teacher then moves toward the updated student distribution on the same student trajectory. This update makes later supervision responsive to the learner and does not require another rollout. On Qwen3-8B in non-thinking mode, DualOPSD improves avg@12 over OPSD by 23.61, 13.89, and 10.00 points on AIME 2024, AIME 2025, and HMMT 2025. Results at 1.7B and 4B show that the accuracy gain depends on model scale. Across all three scales, DualOPSD reduces truncation. The 4B diagnostic also shows lower KL in both directions between the teacher and student.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2608.26019 [cs.LG]
(or arXiv:2608.26019v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.26019 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 241 words
Related coverage
Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning
arXiv · Jun 10, 2026
Function-Space Temporal Self-Distillation for Policy Optimization
arXiv · Oct 15, 2026
Learning from a Thoughtful Teacher: Adaptive On-Policy Self-Distillation for Mathematical Reasoning
arXiv · Sep 29, 2026
RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning
arXiv · Sep 18, 2026