When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillation

Abstract:Top-K teacher logits make on-policy distillation tractable, but probability mass is not the same as decision support. In a two-teacher tool-use setting, vanilla generalized knowledge distillation raises tool-call recall while also calling on examples that require direct answers. The response teacher's top-32 retains 99.99% of its probability mass yet contains the tool-call behavior-switch token on only 0.4% of 1,500 audited prompts; even top-256 covers only 52.2%. Because omitted logits receive zero direct gradient under the truncated objective, the tool teacher reinforces entry while the response teacher usually cannot oppose it. Frozen replay shows that a wrong entry then amplifies divergence along the generated trajectory. Restoring the tool-call token only at the first response position moves first-token entry but mostly delays eventual calls. Restoring it at every response position closes this gap: across three matched seeds, full-generation over-calling falls from 14.2+/-2.1% to 3.7+/-0.5%. The correction is not class-selective: tool-call recall falls from 91.5+/-1.7% to 79.1+/-1.7%, and exact multi-turn success from 7.0+/-0.3% to 3.4+/-0.5%. The matched interventions therefore identify decision-critical support omission as a causal mechanism of the observed drift under the tested teacher-top-32 objective, but restoring it traces a conservatism-capability trade-off rather than a free improvement. We further compare fixed clipping, global reweighting, localized soft compression, and validation-tuned inference-time bias as alternative ways to move this operating point. These results expose a failure mode hidden by near-complete probability-mass coverage and motivate support-aware auditing of compressed distillation.

Computation and Language (cs.CL); Machine Learning (cs.LG)

arXiv:2607.07050 [cs.CL]

(or arXiv:2607.07050v3 [cs.CL] for this version)