analiza aiqbench
The article is squarely about LLM behavior control, so it passes the topic gate. Its primary subject is the robustness of activation steering, a training-free inference-time technique for steering LLM behavior, under adversarial input perturbations. The findings are notable: reported directional robustness drops by up to 64%, and layer selection is fragile, with optimal layers shifting by up to 17 positions under perturbation. Readers building on steering for safety or control should treat it as an unreliable deployment tool until layer selection and vector extraction are hardened, though the study is an arXiv preprint and its results have not been independently replicated.
Adversarial Robustness of Activation Steering in Large Language Models
arXiv:2606.07696v2 Announce Type: replace-cross Abstract: Activation steering has become a popular training-free method to control LLMs' behavior by injecting precomputed direction vectors into the model's residual stream at inference time. Yet, its robustness to realistic input variation remains under-explored. We present the first systematic evaluation of activation steering robustness under adversarial text perturbations on the inputs, covering four extraction methods, three attack strategies, six personas from the Anthropic Model-Written Evaluations dataset, and five models ranging from 1.5B to 30B parameters. Attacks succeed broadly across all settings: directional robustness drops by up to 64\%, post-attack confidence collapses near or below 0.25 across all methods and models, and steering strength degrades on nearly every steerable input. Layer selection is equally fragile, with the optimal layer identified by an automated method on clean inputs shifting by up to 17 positions under perturbation, a failure that compounds the vector-level breakdown. Extracting vectors from adversarially perturbed inputs partially recovers steerability for PCA and MD on mid-to-large models, but they consistently fail to locate the improved optimal layer, limiting the practical benefit of this mitigation. Together, these findings reveal that the brittleness of activation steering is structural rather than method-specific, and that current layer selection strategies are not robust enough for real-world deployment. Our code is available at \href{https://github.com/kien206/steering-adv-robustness}{github.com/kien206/steering-adv-robustness}.
Abstract:Activation steering has become a popular training-free method to control LLMs' behavior by injecting precomputed direction vectors into the model's residual stream at inference time. Yet, its robustness to realistic input variation remains under-explored. We present the first systematic evaluation of activation steering robustness under adversarial text perturbations on the inputs, covering four extraction methods, three attack strategies, six personas from the Anthropic Model-Written Evaluations dataset, and five models ranging from 1.5B to 30B parameters. Attacks succeed broadly across all settings: directional robustness drops by up to 64\%, post-attack confidence collapses near or below 0.25 across all methods and models, and steering strength degrades on nearly every steerable input. Layer selection is equally fragile, with the optimal layer identified by an automated method on clean inputs shifting by up to 17 positions under perturbation, a failure that compounds the vector-level breakdown. Extracting vectors from adversarially perturbed inputs partially recovers steerability for PCA and MD on mid-to-large models, but they consistently fail to locate the improved optimal layer, limiting the practical benefit of this mitigation. Together, these findings reveal that the brittleness of activation steering is structural rather than method-specific, and that current layer selection strategies are not robust enough for real-world deployment. Our code is available at \href{this https URL}{this http URL}.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.07696 [cs.LG]
(or arXiv:2606.07696v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.07696 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 307 cuvinte
Subiecte conexe
Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approach
arXiv · 8 iun. 2026
Geometry Is Not Robustness: A Trajectory-Level Study of PGD Evaluation
arXiv · 19 aug. 2026
Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classification
arXiv · 25 aug. 2026
CARD: Closing the Teacher-Student Loop for Adversarial Robustness Distillation
arXiv · 15 oct. 2026