aiqbench analysis
PocketAgents introduces a manifest-driven framework that constrains autonomous LLMs within strict telemetry boundaries and typed schemas to execute defensive actions like network blocking. By validating these agents against simulated ransomware attacks, the research demonstrates how structured output constraints make AI-driven cybersecurity measurable, extensible, and attributable. This work bridges the gap between generative model flexibility and the rigorous reliability required for real-world cyber defense systems.
PocketAgents: A Manifest-Driven Library of Autonomous Defense Agents
arXiv:2605.21694v2 Announce Type: replace-cross Abstract: Connecting large language models (LLMs) to defensive enforcement requires more than asking a model whether an attack is happening. A defender must decide which model outputs may change the system state, which outputs must be rejected, and how failures should be recorded. We present PocketAgents, a manifest-driven library of autonomous defense agents. Each agent is installed as three data files: a manifest, a prompt, and a runtime context. The shared runtime gives the agent bounded telemetry access and accepts only typed reports whose requested action appears in the manifest. We implemented PocketAgents on top of Perry, a cyber-deception testbed, and evaluated two agents for the Command and Control and Exfiltration tactics in 18 closed-loop trials of a DarkSide-inspired attack on a small enterprise topology. Thirteen trials produced validated network-block actions and contained the attack; four failed schema validation; one produced a valid no-action decision. The experiments show that a typed boundary makes LLM-driven defense measurable, extensible, and attributable.
Abstract:Connecting large language models (LLMs) to defensive enforcement requires more than asking a model whether an attack is happening. A defender must decide which model outputs may change the system state, which outputs must be rejected, and how failures should be recorded. We present PocketAgents, a manifest-driven library of autonomous defense agents. Each agent is installed as three data files: a manifest, a prompt, and a runtime context. The shared runtime gives the agent bounded telemetry access and accepts only typed reports whose requested action appears in the manifest. We implemented PocketAgents on top of Perry, a cyber-deception testbed, and evaluated two agents for the Command and Control and Exfiltration tactics in 18 closed-loop trials of a DarkSide-inspired attack on a small enterprise topology. Thirteen trials produced validated network-block actions and contained the attack; four failed schema validation; one produced a valid no-action decision. The experiments show that a typed boundary makes LLM-driven defense measurable, extensible, and attributable.
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
arXiv:2605.21694 [cs.CR]
(or arXiv:2605.21694v2 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2605.21694 Focus to learn more arXiv-issued DOI via DataCite
https://doi.org/10.5753/sbseg.2026.28577 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 264 words
Related coverage

News
Quantum Computing Report · Jun 7, 2026
Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident
Hugging Face · Jul 27, 2026

[AINews] Meta Muse Wonder & Wonder Full: MSL’s Claude 5.5-tier Frontier Model
Latent Space · Oct 13, 2026

Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
AI Alignment Forum · Aug 3, 2026