analiza aiqbench
This study provides the first mechanistic analysis of pre-training-injected backdoors in Large Language Models, revealing that triggers co-opt existing language-processing circuits rather than forming new ones. By leveraging activation patching across multiple model scales, researchers found significant overlap between trigger-specific and natural language attention heads, offering a pathway for more robust detection methods. These findings advance the field's understanding of adversarial vulnerabilities within LLM internals and inform future interpretability-driven defense strategies.
Language Triggers Hijack Language Circuits: A Mechanistic Analysis of Backdoor Behaviors in Large Language Models
arXiv:2602.10382v3 Announce Type: replace Abstract: Backdoor attacks pose significant security risks for Large Language Models (LLMs), yet the internal mechanisms by which triggers operate remain poorly understood. We present the first mechanistic analysis of trigger-induced language-switching backdoors injected during pre-training, studying the Gaperon model family (1B, 8B and 24B). Using activation patching, we localize trigger formation and identify which attention heads process trigger and natural language information. Our central finding is that trigger heads substantially overlap with heads naturally encoding output language across model scales, with Jaccard indices between 0.18 and 0.43 over the top 10 heads identified. This suggests that backdoor triggers do not form new circuits but instead co-opt the model's existing language components and representations. These findings have implications for backdoor defense as detection methods and mitigation strategies could leverage this entanglement between triggers and natural behaviors. More broadly, our work represents a first step toward a more realistic mechanistic understanding of pre-training-injected backdoors in LLMs, paving the way for principled, interpretability-driven defenses.
Abstract:Backdoor attacks pose significant security risks for Large Language Models (LLMs), yet the internal mechanisms by which triggers operate remain poorly understood. We present the first mechanistic analysis of trigger-induced language-switching backdoors injected during pre-training, studying the Gaperon model family (1B, 8B and 24B). Using activation patching, we localize trigger formation and identify which attention heads process trigger and natural language information. Our central finding is that trigger heads substantially overlap with heads naturally encoding output language across model scales, with Jaccard indices between 0.18 and 0.43 over the top 10 heads identified. This suggests that backdoor triggers do not form new circuits but instead co-opt the model's existing language components and representations. These findings have implications for backdoor defense as detection methods and mitigation strategies could leverage this entanglement between triggers and natural behaviors. More broadly, our work represents a first step toward a more realistic mechanistic understanding of pre-training-injected backdoors in LLMs, paving the way for principled, interpretability-driven defenses.
Computation and Language (cs.CL)
arXiv:2602.10382 [cs.CL]
(or arXiv:2602.10382v3 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 173 cuvinte
Subiecte conexe
Defense Against LLM Backdoors using Critical Neuron Isolation Pruning
arXiv · 23 iul. 2026
Curvature-Guided Module Localization for Low-Rank Detoxification of Backdoored Large Language Models
arXiv · 8 oct. 2026
Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
arXiv · 17 iun. 2026
MemPoison: Bypassing Selective Memory Mechanisms to Plant Backdoors in LLM Agents
arXiv · 29 sept. 2026