analiza aiqbench
This research investigates how much privileged information an AI model needs to effectively teach itself during on-policy self-distillation, challenging the assumption that complete reference solutions are always optimal. By testing various abstraction levels—ranging from full reasoning paths to simple problem categories—the study finds that intermediate contexts can improve performance while significantly reducing token usage. Ultimately, the work suggests that effective self-teaching relies on providing the student with an abstraction level it can act upon, rather than overwhelming it with every possible detail.
What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation
arXiv:2609.25623v1 Announce Type: cross Abstract: More privileged information does not always make a better teacher. We study this tension in on-policy self-distillation (OPSD), where a frozen copy of the base model scores the student's own rollouts under privileged context, conventionally a complete reference solution that bundles the final answer with one particular reasoning path. Holding the student view and training fixed within each scale, we compare that default against three abstractions compiled offline, a named strategy, a method-independent framing, and a problem category, and against an answer-only control that keeps the destination but removes the path. In the primary runs on competition mathematics, the best intermediate contexts improve the in-domain peak mean over the full solution by 1.4 points at 4B and 1.6 at 8B, while storing an order of magnitude fewer hint tokens. Comparisons across three seeds also show positive mean gains for the framing and category contexts at both scales. Answer-only conditioning remains competitive in the primary runs, within 0.2 points of the full solution at these scales. The preferred context varies with student scale and task. Initial teacher-student KL does not order downstream performance. What a self-teacher should see is therefore not everything it could, but the level of abstraction its student can still act on.
Title:What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation
Abstract:More privileged information does not always make a better teacher. We study this tension in on-policy self-distillation (OPSD), where a frozen copy of the base model scores the student's own rollouts under privileged context, conventionally a complete reference solution that bundles the final answer with one particular reasoning path. Holding the student view and training fixed within each scale, we compare that default against three abstractions compiled offline, a named strategy, a method-independent framing, and a problem category, and against an answer-only control that keeps the destination but removes the path. In the primary runs on competition mathematics, the best intermediate contexts improve the in-domain peak mean over the full solution by 1.4 points at 4B and 1.6 at 8B, while storing an order of magnitude fewer hint tokens. Comparisons across three seeds also show positive mean gains for the framing and category contexts at both scales. Answer-only conditioning remains competitive in the primary runs, within 0.2 points of the full solution at these scales. The preferred context varies with student scale and task. Initial teacher-student KL does not order downstream performance. What a self-teacher should see is therefore not everything it could, but the level of abstraction its student can still act on.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2609.25623 [cs.LG]
(or arXiv:2609.25623v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.25623 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 312 cuvinte
Subiecte conexe
Projection sampling preserves capabilities but diminishes gains from alignment fine-tuning
LessWrong · 16 oct. 2026
Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding
arXiv · 29 sept. 2026
Instruct, Not Answer: Using Instruction Privileges in On-Policy Context Distillation
arXiv · 30 sept. 2026
ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation
arXiv · 6 iun. 2026