aiqbench analysis
This research introduces SCOPE-OPSD, a method that enhances on-policy self-distillation by projecting teacher-student residuals onto a Fisher-conditioned subspace derived from language model heads. By leveraging existing forward passes without additional rollouts or inference modules, the approach consistently outperforms standard OPSD and random controls across various Qwen3 model checkpoints. The study validates that utilizing aligned final-layer discrepancies provides a robust second channel for supervision, offering a compact solution for short-budget training scenarios.
SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation
arXiv:2609.12579v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) scores student-generated prefixes with a solution-conditioned self-teacher, yet transfers supervision only through next-token probabilities. We ask whether the aligned final-layer discrepancy offers a useful second channel, and how to test that channel without confusing its geometry with auxiliary strength. SCOPE-OPSD projects the privileged teacher-student residual onto a frozen rank-64 factor estimated from residual covariance and language-model-head Fisher sensitivity. It reuses the forwards already required by OPSD and adds neither rollouts nor inference-time modules. A matched Random control preserves the structured factor's rank and nonzero spectrum and uses per-arm gradient-RMS calibration, isolating the effect of the data-dependent orientation. Across the complete 25/50/75/100-step trajectories for Qwen3-1.7B, 4B, and 8B, Structured is never below Pure OPSD, with strict gains in 11 of the 12 model-checkpoint combinations and an exact tie at 4B step 25. Structured also exceeds matched Random in 10 of the 12 combinations. At step 75 on Qwen3-1.7B, Structured exceeds matched Random by 1.39 Macro Avg@12 points in each of two independent training reruns. A cross-fitted diagnostic also shows 4.40 times greater held-out privileged-gap capture than the matched random orientation. The results support a compact, Fisher-conditioned privileged subspace for short-budget OPSD.
Authors:Yunmeng Chen (1), Kunyu Wang (2), Peihan Li (1), Yi Wang (1), Shuyin Xia (3), Yi Liu (1), Xinyong Cheng (2), Dehui Wang (2), Xiangyong Zhai (2), Yanxing Liu (1), Song Liu (1) ((1) Chongqing Ant Consumer Finance Co., Ltd., (2) Alibaba Cloud Computing Co., Ltd., (3) Chongqing University of Posts and Telecommunications)
Abstract:On-policy self-distillation (OPSD) scores student-generated prefixes with a solution-conditioned self-teacher, yet transfers supervision only through next-token probabilities. We ask whether the aligned final-layer discrepancy offers a useful second channel, and how to test that channel without confusing its geometry with auxiliary strength. SCOPE-OPSD projects the privileged teacher-student residual onto a frozen rank-64 factor estimated from residual covariance and language-model-head Fisher sensitivity. It reuses the forwards already required by OPSD and adds neither rollouts nor inference-time modules. A matched Random control preserves the structured factor's rank and nonzero spectrum and uses per-arm gradient-RMS calibration, isolating the effect of the data-dependent orientation. Across the complete 25/50/75/100-step trajectories for Qwen3-1.7B, 4B, and 8B, Structured is never below Pure OPSD, with strict gains in 11 of the 12 model-checkpoint combinations and an exact tie at 4B step 25. Structured also exceeds matched Random in 10 of the 12 combinations. At step 75 on Qwen3-1.7B, Structured exceeds matched Random by 1.39 Macro Avg@12 points in each of two independent training reruns. A cross-fitted diagnostic also shows 4.40 times greater held-out privileged-gap capture than the matched random orientation. The results support a compact, Fisher-conditioned privileged subspace for short-budget OPSD.
16 pages, 3 figures. Yunmeng Chen, Kunyu Wang, and Peihan Li contributed equally. Corresponding author: Song Liu
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2609.12579 [cs.LG]
(or arXiv:2609.12579v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.12579 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 284 words
Related coverage
PreSO: Privileged Representation Isolation for On-Policy Self-Distillation
arXiv · Oct 15, 2026
One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation
arXiv · Aug 27, 2026
Projection sampling preserves capabilities but diminishes gains from alignment fine-tuning
LessWrong · Oct 16, 2026
Learn How to Act from Your Own Interactions: On-Policy Self-Distillation for GUI Agents
arXiv · Sep 24, 2026