aiqbench analysis
The paper proposes an attacker-defender reinforcement learning setup to harden LLM agents against malicious content hidden in queries, tool descriptions, or environment observations. Its main claim is improved adversarial robustness without a loss in general capability, but that rests on the authors' own benchmarks, so independent replication matters. For readers building tool-using agents, the useful signal is that co-evolved adversarial training is being extended beyond language-level safety to multi-turn tool use.
AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety
arXiv:2610.19422v1 Announce Type: new Abstract: Large language model (LLM) agents accomplish complex tasks through multi-turn interactions with external tools. However, malicious content embedded in user queries, tool descriptions, or environment observations can manipulate their decisions and induce unsafe actions. Adversarial co-evolution has emerged as an effective approach to strengthening LLM safety, but existing work primarily focuses on language-level safety or indirect prompt injection in web tasks. In this paper, we introduce AgentSafeEvo, a co-evolutionary reinforcement learning framework for agent safety built on an attacker-defender game spanning diverse tasks and attack types. The attacker learns to construct adversarial agent tasks that expose weaknesses in the defender's tool use, while the defender learns to complete tasks safely under these attacks. We further introduce learnability-guided task weighting, which uses within-task defender-reward variability to adjust each task's contribution to attacker and defender updates. Extensive experiments on agent safety and general capability benchmarks demonstrate that our method improves adversarial robustness while maintaining general agent capabilities.
Abstract:Large language model (LLM) agents accomplish complex tasks through multi-turn interactions with external tools. However, malicious content embedded in user queries, tool descriptions, or environment observations can manipulate their decisions and induce unsafe actions. Adversarial co-evolution has emerged as an effective approach to strengthening LLM safety, but existing work primarily focuses on language-level safety or indirect prompt injection in web tasks. In this paper, we introduce AgentSafeEvo, a co-evolutionary reinforcement learning framework for agent safety built on an attacker-defender game spanning diverse tasks and attack types. The attacker learns to construct adversarial agent tasks that expose weaknesses in the defender's tool use, while the defender learns to complete tasks safely under these attacks. We further introduce learnability-guided task weighting, which uses within-task defender-reward variability to adjust each task's contribution to attacker and defender updates. Extensive experiments on agent safety and general capability benchmarks demonstrate that our method improves adversarial robustness while maintaining general agent capabilities.
Artificial Intelligence (cs.AI)
arXiv:2610.19422 [cs.AI]
(or arXiv:2610.19422v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.19422 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 249 words
Related coverage
SEABench: Benchmarking Endogenous Misalignment In Self-Evolving Agents
arXiv · Sep 30, 2026
The Alignment Problem: Computer Science Meets Analytic Philosophy
AI Alignment Forum · Oct 18, 2026
HASTE: Evolving Agent Harnesses Against Emerging Attacks Using Sparse Evidence
arXiv · Oct 6, 2026
EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM Agents
arXiv · Jun 3, 2026