aiqbench analysis
This research identifies a "narrow safety basin" in LLM parameter spaces where updates orthogonal to the alignment direction rapidly degrade safety, while those along the direction preserve it. The proposed AsFT method constrains fine-tuning updates within this basin, successfully reducing harmful behaviors by up to 7.60% while simultaneously improving task performance. This approach offers a mathematically grounded strategy for maintaining model alignment without sacrificing capabilities during the fine-tuning process.
AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basin
arXiv:2506.08473v4 Announce Type: replace Abstract: Fine-tuning large language models (LLMs) improves performance but introduces critical safety vulnerabilities: even minimal harmful data can severely compromise safety measures. We observe that perturbations orthogonal to the alignment direction - defined by weight differences between aligned (safe) and unaligned models - rapidly compromise model safety. In contrast, updates along the alignment direction largely preserve it, revealing the parameter space as a "narrow safety basin". To address this, we propose AsFT (Anchoring Safety in Fine-Tuning) to maintain safety by explicitly constraining update directions during fine-tuning. By penalizing updates orthogonal to the alignment direction, AsFT effectively constrains the model within the "narrow safety basin," thus preserving its inherent safety. Extensive experiments on multiple datasets and models show that AsFT reduces harmful behaviors by up to 7.60%, improves task performance by 3.44%, and consistently outperforms existing methods across multiple tasks.
Abstract:Fine-tuning large language models (LLMs) improves performance but introduces critical safety vulnerabilities: even minimal harmful data can severely compromise safety measures. We observe that perturbations orthogonal to the alignment direction - defined by weight differences between aligned (safe) and unaligned models - rapidly compromise model safety. In contrast, updates along the alignment direction largely preserve it, revealing the parameter space as a "narrow safety basin". To address this, we propose AsFT (Anchoring Safety in Fine-Tuning) to maintain safety by explicitly constraining update directions during fine-tuning. By penalizing updates orthogonal to the alignment direction, AsFT effectively constrains the model within the "narrow safety basin," thus preserving its inherent safety. Extensive experiments on multiple datasets and models show that AsFT reduces harmful behaviors by up to 7.60%, improves task performance by 3.44%, and consistently outperforms existing methods across multiple tasks.
Machine Learning (cs.LG)
arXiv:2506.08473 [cs.LG]
(or arXiv:2506.08473v4 [cs.LG] for this version)
Full text under the publisher's open licence · 149 words
Related coverage
Personas in a Pre-Internet Language Model
LessWrong · Oct 13, 2026
The Continual Constitutional Convention: How language models revise the documents that govern them
AI Alignment Forum · Oct 13, 2026
Building and evaluating model diffing agents
AI Alignment Forum · Jun 12, 2026
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · Sep 18, 2026