aiqbench analysis
This research investigates how learning rate dynamics during supervised fine-tuning can inadvertently trigger catastrophic forgetting in large language models, ultimately leading to performance degradation known as overtraining. By linking optimization sharpness with decay schedules, the study provides a mechanistic explanation for why certain training strategies erode fundamental capabilities acquired during pre-training. These findings are critical for refining post-training pipelines and preserving model utility across diverse tasks.
(How) Learning Rates Regulate Catastrophic Overtraining
arXiv:2604.13627v2 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) is a common first stage of LLM post-training, teaching the model to follow instructions and shaping its behavior as a helpful assistant. At the same time, SFT may harm the fundamental capabilities of an LLM, particularly after long pretraining: a phenomenon known as catastrophic overtraining (Springer et al., 2025). To understand overtraining, we first investigate catastrophic forgetting in finetuning through the lens of implicit regularization of the learning rate. For models trained to the same SFT loss, we identify how the learning rate mediates optimization: finetuning with large and small steps converges to qualitatively different models. Next, we link forgetting to overtraining: learning rate decay increases the sharpness of the pretrained model, which in turn exacerbates catastrophic forgetting during SFT, leading to overtraining. Our findings paint a picture of the overtraining mechanism in LLMs and broadly contribute to the understanding of the interplay between optimization dynamics during pretraining and finetuning.
Abstract:Supervised fine-tuning (SFT) is a common first stage of LLM post-training, teaching the model to follow instructions and shaping its behavior as a helpful assistant. At the same time, SFT may harm the fundamental capabilities of an LLM, particularly after long pretraining: a phenomenon known as catastrophic overtraining (Springer et al., 2025). To understand overtraining, we first investigate catastrophic forgetting in finetuning through the lens of implicit regularization of the learning rate. For models trained to the same SFT loss, we identify how the learning rate mediates optimization: finetuning with large and small steps converges to qualitatively different models. Next, we link forgetting to overtraining: learning rate decay increases the sharpness of the pretrained model, which in turn exacerbates catastrophic forgetting during SFT, leading to overtraining. Our findings paint a picture of the overtraining mechanism in LLMs and broadly contribute to the understanding of the interplay between optimization dynamics during pretraining and finetuning.
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2604.13627 [cs.LG]
(or arXiv:2604.13627v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2604.13627 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 249 words
Related coverage
The Alignment Problem: Computer Science Meets Analytic Philosophy
AI Alignment Forum · Oct 18, 2026
OpenRouter: from Seed to Stripe — with OpenRouter’s Alex Atallah & AMP’s Anjney Midha
Latent Space · Sep 25, 2026
The art and science of hyperparameter optimization on Amazon Nova Forge
AWS · Jun 2, 2026
CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance
arXiv · Jul 31, 2026