analiza aiqbench
This paper is clearly about AI: sparse and continual fine-tuning of language models using reinforcement learning, with the core ideas centered on attention-head ranking by gradient norm and head partitioning across tasks. It falls under Machine Learning, since the contribution is a training method and its theoretical and empirical analysis rather than a product or policy. For practitioners, the useful takeaway is that updating a small, well-chosen subset of attention heads may match denser fine-tuning while limiting forgetting, though the results depend on the specific models and benchmarks tested.
Ranking and Partitioning of Attention Heads for Sparse and Continual Reinforcement Learning Fine-tuning
arXiv:2610.21336v1 Announce Type: cross Abstract: We study sparse and continual reinforcement learning fine-tuning of language models through attention-head ranking and partitioning. For sparse learning, we rank attention heads by gradient norm to identify a small subset of heads that can be trained effectively. For continual learning, we partition heads across tasks to reduce cross-task interference and preserve previously learned capabilities. Theoretically, we relate gradient norm to training effectiveness, and further characterize why sparse, separated head updates can remain stable. Empirically, across math, instruction-following, and coding tasks, tuning higher-ranked heads consistently outperforms tuning lower-ranked ones while often matching denser baselines, and head partitioning improves retention while preserving adaptation to new tasks.
Abstract:We study sparse and continual reinforcement learning fine-tuning of language models through attention-head ranking and partitioning. For sparse learning, we rank attention heads by gradient norm to identify a small subset of heads that can be trained effectively. For continual learning, we partition heads across tasks to reduce cross-task interference and preserve previously learned capabilities. Theoretically, we relate gradient norm to training effectiveness, and further characterize why sparse, separated head updates can remain stable. Empirically, across math, instruction-following, and coding tasks, tuning higher-ranked heads consistently outperforms tuning lower-ranked ones while often matching denser baselines, and head partitioning improves retention while preserving adaptation to new tasks.
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2610.21336 [cs.LG]
(or arXiv:2610.21336v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.21336 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 203 cuvinte
Subiecte conexe
CoWindow Attention: Full Causal Coverage Is a Collective Property
arXiv · 29 sept. 2026
GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model
Meta Engineering · 3 aug. 2026
Building Blocks for Foundation Model Training and Inference on AWS
Hugging Face · 11 mai 2026
[AINews] not much happened today
Latent Space · 3 oct. 2026