aiqbench analysis
This research presents Supervised Transcoder Replacement (STR), a technique designed to mitigate the side effects of model steering in large language models like Gemma and Llama. By learning a replacement for the MLP computation at the steering layer, STR effectively reduces degradation of non-target behaviors while maintaining control over specific target actions. The method demonstrates significant improvements in out-of-distribution safety testing, lowering attack success rates without requiring protection objectives during the initial fitting phase.
STR: Supervised Transcoder Replacement for Reducing Steering Side Effects
arXiv:2609.32519v1 Announce Type: cross Abstract: Model steering can strengthen a target behavior while degrading other useful behaviors. We introduce Supervised Transcoder Replacement (STR) to reduce these side effects for existing steering methods, including those fitted without a protection objective. STR learns a replacement for the multilayer perceptron (MLP) computation at the steering layer through supervision for target control, non-target preservation, and fidelity without steering. Selected steering methods then fit directions on the frozen replacement while retaining their own fitting objectives. We evaluate three steering methods across Gemma and Llama models using Corrigibility preferences and four harmful-request safety datasets. SALAD-Bench supplies protection training data and a separate in-distribution evaluation split; HarmBench, AdvBench, and StrongREJECT are reserved for out-of-distribution testing. STR substantially reduces steering side effects on the in-distribution evaluation and extends this protection to the unseen safety datasets while retaining effective target control. For target-only supervised steering vectors, pooled out-of-distribution attack success rate falls from 42.46% to 14.42% on Gemma-3-4B and from 34.97% to 12.91% on Gemma-3-12B. These results show that replacement training can benefit steering methods fitted without protection objectives.
Abstract:Model steering can strengthen a target behavior while degrading other useful behaviors. We introduce Supervised Transcoder Replacement (STR) to reduce these side effects for existing steering methods, including those fitted without a protection objective. STR learns a replacement for the multilayer perceptron (MLP) computation at the steering layer through supervision for target control, non-target preservation, and fidelity without steering. Selected steering methods then fit directions on the frozen replacement while retaining their own fitting objectives. We evaluate three steering methods across Gemma and Llama models using Corrigibility preferences and four harmful-request safety datasets. SALAD-Bench supplies protection training data and a separate in-distribution evaluation split; HarmBench, AdvBench, and StrongREJECT are reserved for out-of-distribution testing. STR substantially reduces steering side effects on the in-distribution evaluation and extends this protection to the unseen safety datasets while retaining effective target control. For target-only supervised steering vectors, pooled out-of-distribution attack success rate falls from 42.46% to 14.42% on Gemma-3-4B and from 34.97% to 12.91% on Gemma-3-12B. These results show that replacement training can benefit steering methods fitted without protection objectives.
Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2609.32519 [cs.AI]
(or arXiv:2609.32519v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.32519 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 272 words
Related coverage
Why I Left Google DeepMind
LessWrong · Jul 15, 2026
Inoculate or Reflect? Two training interventions under prompting, steering, and patching
LessWrong · Jul 26, 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026
[AINews] Claude Opus 5.5, the new default model for AINews — and everybody cuts prices 40-50%
Latent Space · Sep 23, 2026