analiza aiqbench
This paper is squarely about AI/ML: it proposes a method for multi-teacher on-policy distillation of language models, combining task-arithmetic-style parameter merging with trainable student weights. The core claim is that treating expert models as parameter priors, rather than fixed merge endpoints, speeds up recovery of expert capabilities and reaches higher final scores (94.7% vs. 80.9% for the baseline at 1.7B). These are single-paper results on a limited set of domains and model sizes, so readers should wait for independent replication before treating the gains as general, though the approach is relevant to anyone building compact, multi-skill models.
P$^2$-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors
arXiv:2611.03953v1 Announce Type: new Abstract: Multi-teacher on-policy distillation consolidates expert capabilities by supervising student-generated trajectories, whereas task arithmetic offers a training-free alternative through linear parameter combinations. We view the linear combination of student and expert parameters as a single differentiable model and propose P$^2$-MOPD, which jointly optimizes the mixing coefficients and the student weights under the same on-policy distillation objective. Unlike task arithmetic, which stays within the span of the frozen student and expert parameters for any choice of merging coefficients, P$^2$-MOPD keeps the student trainable, so the experts act as parameter priors without confining it to this span. Across four domains at 1.7B and 4B scales, P$^2$-MOPD achieves faster and more balanced recovery than MOPD from the same initialization, surpassing its final average recovery in half the training steps. At the end of training, our 1.7B model reaches 94.7% average recovery, versus 80.9% for MOPD.
Abstract:Multi-teacher on-policy distillation consolidates expert capabilities by supervising student-generated trajectories, whereas task arithmetic offers a training-free alternative through linear parameter combinations. We view the linear combination of student and expert parameters as a single differentiable model and propose P$^2$-MOPD, which jointly optimizes the mixing coefficients and the student weights under the same on-policy distillation objective. Unlike task arithmetic, which stays within the span of the frozen student and expert parameters for any choice of merging coefficients, P$^2$-MOPD keeps the student trainable, so the experts act as parameter priors without confining it to this span. Across four domains at 1.7B and 4B scales, P$^2$-MOPD achieves faster and more balanced recovery than MOPD from the same initialization, surpassing its final average recovery in half the training steps. At the end of training, our 1.7B model reaches 94.7% average recovery, versus 80.9% for MOPD.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2611.03953 [cs.LG]
(or arXiv:2611.03953v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2611.03953 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 238 cuvinte