aiqbench analysis
This study compares two distinct intervention methods for steering large language models to refuse harmful requests, evaluating the efficacy of Iterative Nullspace Projection (INLP) against established difference-in-means techniques. The findings suggest that while INLP's counterfactual flipping approach offers competitive suppression capabilities with greater tunability, its geometric mechanism differs fundamentally from directional ablation by manipulating activation clusters rather than collapsing them. These insights provide researchers with a clearer understanding of the internal geometry required to safely manipulate model behavior without degrading baseline performance.
Refusal Beyond a Single Direction: A Preliminary Comparison of Diff-in-Means and INLP
arXiv:2606.13720v2 Announce Type: replace Abstract: Arditi et al. (2024) has shown that refusal in safety fine-tuned chat models is mediated by a single linear direction in the residual stream, recoverable by a difference-in-means (DiM) of harmful and harmless activations. We compare DiM-based interventions (activation addition and directional ablation) with two interventions derived from Iterative Nullspace Projection (INLP)-nullspace projection and counterfactual flipping-on five open-weight chat models, asking whether INLP can match DiM at steering refusal and whether its richer parameterisation yields more tweakable interventions. INLP counterfactual flipping is competitive with DiM directional ablation on refusal suppression, while nullspace projection is weaker on most models. Applying each method at the layer selected by the other shows that flipping's competitiveness is robust to this change while directional ablation's is not, suggesting that DiM's apparent edge is partly a property of its own selection procedure. Restricting INLP to the leading directions of the extracted subspace preserves most of the suppression effect at near-baseline perplexity, giving a tunable capability. Geometrically, the two INLP interventions land in qualitatively different regions of activation space: nullspace projection collapses transformed activations between the harmful and harmless clusters, while counterfactual flipping moves them into the opposite cluster, suggesting that the model encodes the absence of a concept differently from its opposit--an intriguing distinction that warrants further investigation in future work.
Abstract:Arditi et al. (2024) has shown that refusal in safety fine-tuned chat models is mediated by a single linear direction in the residual stream, recoverable by a difference-in-means (DiM) of harmful and harmless activations. We compare DiM-based interventions (activation addition and directional ablation) with two interventions derived from Iterative Nullspace Projection (INLP)-nullspace projection and counterfactual flipping-on five open-weight chat models, asking whether INLP can match DiM at steering refusal and whether its richer parameterisation yields more tweakable interventions. INLP counterfactual flipping is competitive with DiM directional ablation on refusal suppression, while nullspace projection is weaker on most models. Applying each method at the layer selected by the other shows that flipping's competitiveness is robust to this change while directional ablation's is not, suggesting that DiM's apparent edge is partly a property of its own selection procedure. Restricting INLP to the leading directions of the extracted subspace preserves most of the suppression effect at near-baseline perplexity, giving a tunable capability. Geometrically, the two INLP interventions land in qualitatively different regions of activation space: nullspace projection collapses transformed activations between the harmful and harmless clusters, while counterfactual flipping moves them into the opposite cluster, suggesting that the model encodes the absence of a concept differently from its opposit--an intriguing distinction that warrants further investigation in future work.
Artificial Intelligence (cs.AI)
arXiv:2606.13720 [cs.AI]
(or arXiv:2606.13720v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.13720 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 307 words
Related coverage
Refusal Beyond a Single Direction: A Preliminary Comparison of Diff-in-Means and INLP
arXiv · Jun 15, 2026
The Continual Constitutional Convention: How language models revise the documents that govern them
AI Alignment Forum · Oct 13, 2026
Construct validity of Claude Opus 4.8's System Card – A commentary
LessWrong · Jun 12, 2026
The Entangled Dimensions of Decision Theory
LessWrong · Jul 30, 2026