aiqbench analysis
This research introduces Latent Fusion Jailbreak (LFJ), a white-box attack that manipulates safety-aligned LLMs by interpolating hidden states between harmful and benign queries to bypass refusal mechanisms. The study demonstrates an 94% success rate across multiple models but also proposes latent adversarial training as a defense, reducing the attack's effectiveness significantly while noting limitations in transferability and utility preservation.
Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs
arXiv:2508.10029v3 Announce Type: replace-cross Abstract: Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations. We introduce Latent Fusion Jailbreak (LFJ), which works by pairing a harmful query with a structurally similar but benign counterpart, then interpolating their hidden states at carefully selected layers and token positions. Refusal-loss gradients determine exactly where to intervene, and we optimise layer-wise mixing coefficients using token-normalised compliance and refusal-suppression objectives. The edited prompt states propagate sequentially through the remaining transformer blocks. Across four safety benchmarks and five open-weight target models, LFJ reaches a macro-averaged attack success rate (ASR) of 94.13% under the white-box protocol we describe. Because LFJ directly accesses internal states, comparisons with prompt-only attacks serve as a descriptive reference rather than a matched evaluation. Dropping rejection sampling lowers ASR to 86.72%, whereas replacing the structured harmful-benign pairing with random pairing causes it to fall to 27.45%. We also design an LFJ-specific latent adversarial training procedure that, when the attack is re-optimised against the defended model, reduces ASR from 94.13% to 12.37%. This defence evaluation does not cover transfer to other attack types or preservation of benign utility.
Abstract:Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations. We introduce Latent Fusion Jailbreak (LFJ), which works by pairing a harmful query with a structurally similar but benign counterpart, then interpolating their hidden states at carefully selected layers and token positions. Refusal-loss gradients determine exactly where to intervene, and we optimise layer-wise mixing coefficients using token-normalised compliance and refusal-suppression objectives. The edited prompt states propagate sequentially through the remaining transformer blocks. Across four safety benchmarks and five open-weight target models, LFJ reaches a macro-averaged attack success rate (ASR) of 94.13% under the white-box protocol we describe. Because LFJ directly accesses internal states, comparisons with prompt-only attacks serve as a descriptive reference rather than a matched evaluation. Dropping rejection sampling lowers ASR to 86.72%, whereas replacing the structured harmful-benign pairing with random pairing causes it to fall to 27.45%. We also design an LFJ-specific latent adversarial training procedure that, when the attack is re-optimised against the defended model, reduces ASR from 94.13% to 12.37%. This defence evaluation does not cover transfer to other attack types or preservation of benign utility.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)
arXiv:2508.10029 [cs.CL]
(or arXiv:2508.10029v3 [cs.CL] for this version)
Full text under the publisher's open licence · 205 words
Related coverage
Story Imprinting: AI Assistants Absorb Traits from Human Characters They Resemble
LessWrong · Sep 21, 2026
When Behavioral Safety Evaluation Fails: A Representation-Level Perspective
arXiv · Aug 6, 2026
Does Your LLM Trust You?
LessWrong · Aug 4, 2026
Self-Modeling Interventions Modulate Emergent Misalignment
LessWrong · Oct 5, 2026