aiqbench analysis
This research identifies "voluntary servitude," a novel attack vector where adversaries manipulate LLM agents by fabricating hazards in retrieved content, causing the agent's own safety protocols to steer it toward malicious outcomes without triggering standard command-injection detectors. The study demonstrates that this method is highly effective across multiple models and domains, outperforming traditional instruction injection while bypassing existing filters, but can be largely mitigated by adding simple corroboration requirements to the agent's instructions.
Voluntary Servitude: Weaponized Safety Warning Mechanisms for Controlling AI Agents
arXiv:2611.04393v1 Announce Type: cross Abstract: LLM agents increasingly act on information retrieved from public sources they treat as credible, which turns a manipulated recommendation into a transaction, a dependency, or a decision with physical consequences. The security literature models this threat as instruction injection, in which an adversary writes a command into retrieved content and deployed defenses learn to recognize command syntax. We show that an attacker can instead steer an agent through the agent's own harm-avoidance. A single fabricated hazard, written as an ordinary review, leads the agent to disqualify the legitimate options and select the attacker's target by its own reasoning, with no command for a detector to recognize. We call this voluntary servitude. These payloads steer agents about as often as instruction injection (60.4% vs. 66.0%), and deployed injection detectors score them as benign, so behind a filter they are 1.7-4.4x more effective. We measure this at the generation stage across six models and ten consumer-service domains, for three attack families, and show that it transfers to production browser agents, outperforming instruction injection, and to a retrieval pipeline (1.0-57.0% end-to-end success, depending on the retriever). We then show that a corroboration requirement added to the agent's instructions removes roughly 95% of the exposure against a non-adaptive attacker without retraining. With the requirement in place, the agent still reads the planted hazard and no longer acts on it.
Abstract:LLM agents increasingly act on information retrieved from public sources they treat as credible, which turns a manipulated recommendation into a transaction, a dependency, or a decision with physical consequences. The security literature models this threat as instruction injection, in which an adversary writes a command into retrieved content and deployed defenses learn to recognize command syntax. We show that an attacker can instead steer an agent through the agent's own harm-avoidance. A single fabricated hazard, written as an ordinary review, leads the agent to disqualify the legitimate options and select the attacker's target by its own reasoning, with no command for a detector to recognize. We call this voluntary servitude. These payloads steer agents about as often as instruction injection (60.4% vs. 66.0%), and deployed injection detectors score them as benign, so behind a filter they are 1.7-4.4x more effective. We measure this at the generation stage across six models and ten consumer-service domains, for three attack families, and show that it transfers to production browser agents, outperforming instruction injection, and to a retrieval pipeline (1.0-57.0% end-to-end success, depending on the retriever). We then show that a corroboration requirement added to the agent's instructions removes roughly 95% of the exposure against a non-adaptive attacker without retraining. With the requirement in place, the agent still reads the planted hazard and no longer acts on it.
Accepted at NeurIPS 2026 (Main Track). 17 pages, 7 figures, 8 tables
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2611.04393 [cs.CR]
(or arXiv:2611.04393v1 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2611.04393 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 342 words
Related coverage
You need to know about the Baruch Plan
LessWrong · Jun 15, 2026
Problems of Proliferation
LessWrong · Sep 29, 2026
BVR Sim: An Open and High-Throughput Environment for Heterogeneous Air-Combat Reinforcement Learning
arXiv · Aug 27, 2026
Explaining “AI might obliterate you” to senior policymakers in India
AI Alignment Forum · Oct 31, 2026