analiza aiqbench
This study reveals a critical divergence between the functional durability and mechanistic stability of embedded activation steering in language models. While fine-tuning processes like SFT can significantly degrade behavioral effects—such as refusal suppression—the underlying weight edits remain largely intact, suggesting that standard retraining does not dismantle alignment mechanisms but rather overwrites their expression. Consequently, practitioners must implement rigorous post-deployment behavioral validation whenever models undergo downstream training to ensure alignment persists despite functional regression.
Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal
arXiv:2608.24988v1 Announce Type: cross Abstract: Activation steering can be embedded directly into a language model's weights, shaping behaviour without inference-time intervention and offering a way to encode alignment prior to release. However, models are routinely fine-tuned after deployment, and it is unknown whether embedded interventions survive this. We study the stability of embedded steering for refusal suppression and brevity induction across five instruction-tuned models (3B-14B) under non-adversarial SFT and RLHF. Behaviourally, preservation tracks the training data: steering degrades when optimisation pressure contradicts the targeted behaviour and persists otherwise, with refusal ablation losing 64% of its effect on average under SFT. Mechanistically, however, the weight edit survives almost untouched even where behaviour reverts: mean vector recovery is $\rho = 0.004$, and the fine-tuning update along the steering direction is near-orthogonal to its pre-edit weight pattern (mean $\cos\theta = 0.074$). When steered behaviour degrades, fine-tuning does not achieve it by dismantling or reversing the steering mechanism itself. Embedded steering is therefore mechanistically durable but functionally vulnerable, and requires behavioural re-validation after downstream training.
Title:Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal
Abstract:Activation steering can be embedded directly into a language model's weights, shaping behaviour without inference-time intervention and offering a way to encode alignment prior to release. However, models are routinely fine-tuned after deployment, and it is unknown whether embedded interventions survive this. We study the stability of embedded steering for refusal suppression and brevity induction across five instruction-tuned models (3B-14B) under non-adversarial SFT and RLHF. Behaviourally, preservation tracks the training data: steering degrades when optimisation pressure contradicts the targeted behaviour and persists otherwise, with refusal ablation losing 64% of its effect on average under SFT. Mechanistically, however, the weight edit survives almost untouched even where behaviour reverts: mean vector recovery is $\rho = 0.004$, and the fine-tuning update along the steering direction is near-orthogonal to its pre-edit weight pattern (mean $\cos\theta = 0.074$). When steered behaviour degrades, fine-tuning does not achieve it by dismantling or reversing the steering mechanism itself. Embedded steering is therefore mechanistically durable but functionally vulnerable, and requires behavioural re-validation after downstream training.
Accepted at EMNLP 2026 Main. Earlier version at ICLR 2026 Re^4-Align Workshop
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2608.24988 [cs.CL]
(or arXiv:2608.24988v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.24988 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 289 cuvinte
Subiecte conexe
Some Intuitions on Steering Vectors
LessWrong · 29 sept. 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · 8 oct. 2026
The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodology
LessWrong · 23 iul. 2026
Reference-Grafting Matches Fine-Tuning at Eliciting Sandbagged Capabilities
arXiv · 1 sept. 2026