analiza aiqbench
This research identifies a critical vulnerability in LLM agents where adversarial content injected into external observations can persist within the agent's state, remaining dormant until activated by a subsequent benign query. The study formalizes this "Sleeper Attack" threat and demonstrates that even advanced models remain susceptible to such delayed harm across session context, memory, and reusable skills. For developers deploying autonomous agents, this highlights the urgent need for robust mechanisms to detect and purge latent adversarial data from persistent agent states.
Plant, Persist, Trigger: Sleeper Attack on Large Language Model Agents
arXiv:2605.28201v2 Announce Type: replace Abstract: Large Language Model (LLM) agents remain vulnerable to safety threats from the external environment, where attackers inject adversarial content into external observations such as tool-returned data, webpages, or MCP context, causing harmful agentic behaviors such as unsafe actions or incorrect outputs. Existing studies typically focus on single-interaction attacks, where the agent observes adversarial content and immediately exhibits harmful behavior within one user request. However, we show that adversarial content can also persist across interactions served by the same agent, making such threats harder to detect and mitigate. Specifically, adversarial content may persist in the agent state, remain dormant across interactions, and later be activated by a benign user query. We formalize this type of safety threat as Sleeper Attack. To evaluate it, we construct a benchmark with 1,896 instances covering six real-world harmful outcomes, three attack strategies, and three agent state targets: session context, memory, and reusable skills. Experiments on seven strong open-source and closed-source LLMs show that state-of-the-art LLM agents remain vulnerable to Sleeper Attack, even when they achieve low attack success rates under a single-interaction baseline. Our code and data are available at https://anonymous.4open.science/r/skdvnfu23ihr9wdscnksf1asdffsaef.
Abstract:Large Language Model (LLM) agents remain vulnerable to safety threats from the external environment, where attackers inject adversarial content into external observations such as tool-returned data, webpages, or MCP context, causing harmful agentic behaviors such as unsafe actions or incorrect outputs. Existing studies typically focus on single-interaction attacks, where the agent observes adversarial content and immediately exhibits harmful behavior within one user request. However, we show that adversarial content can also persist across interactions served by the same agent, making such threats harder to detect and mitigate. Specifically, adversarial content may persist in the agent state, remain dormant across interactions, and later be activated by a benign user query. We formalize this type of safety threat as Sleeper Attack. To evaluate it, we construct a benchmark with 1,896 instances covering six real-world harmful outcomes, three attack strategies, and three agent state targets: session context, memory, and reusable skills. Experiments on seven strong open-source and closed-source LLMs show that state-of-the-art LLM agents remain vulnerable to Sleeper Attack, even when they achieve low attack success rates under a single-interaction baseline. Our code and data are available at this https URL.
Artificial Intelligence (cs.AI)
arXiv:2605.28201 [cs.AI]
(or arXiv:2605.28201v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2605.28201 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 280 cuvinte
Subiecte conexe
PermaFrost-Attack: Stealth Pretraining Seeding(SPS) for planting Logic Landmines During LLM Training
arXiv · 15 oct. 2026
MemPoison: Bypassing Selective Memory Mechanisms to Plant Backdoors in LLM Agents
arXiv · 29 sept. 2026
Links #3: 2026/06 Part 1
LessWrong · 15 iun. 2026
How might continual learning affect safety and alignment?
LessWrong · 13 iun. 2026