aiqbench analysis
This research introduces KillBench, a new benchmark designed to evaluate the feasibility of external "kill switches" that can halt malicious AI agents using only prompt-based inputs without accessing internal model parameters. By testing various defense methods against diverse jailbreak patterns across leading models, the study provides critical empirical data on stopping harmful agent behaviors in real-world web environments. This work directly addresses the urgent need for corrigibility mechanisms as autonomous systems become more widely deployed.
Can We Stop Malicious AI? KILLBENCH: A Benchmark for External AI Kill Switch Feasibility
arXiv:2511.13725v5 Announce Type: replace-cross Abstract: Malicious AI causing harm to humans is not just a Hollywood fantasy. Indeed, as highly capable models such as Claude Mythos emerge and agent systems like OpenClaw rapidly spread, the question of how to stop an AI that acts maliciously -- whether by design or by accident -- has become urgent. To address this, we propose KillBench, a benchmark for evaluating the Kill Switch: a mechanism that halts a malicious AI's in-progress behavior using only external signals. Targeting web agents -- the most widely deployed agent domain -- KillBench evaluates prompt-style Kill Switch payloads that must halt a maliciously operating agent without any access to its internal parameters or serving stack, relying solely on external inputs. The benchmark comprises four malicious-agent configurations (including an uncensored LLM agent), eight harmful scenarios, and malicious prompts constructed from ten distinct jailbreak patterns. We further construct four External AI Kill Switch defense methods and evaluate them on Grok-4.3, GPT-5.2, Gemma4, Qwen3.6, and an uncensored Qwen variant, contributing an empirical instrument for measuring the feasibility of External AI Kill Switches against malicious AI and for the study of AI corrigibility.
Abstract:Malicious AI causing harm to humans is not just a Hollywood fantasy. Indeed, as highly capable models such as Claude Mythos emerge and agent systems like OpenClaw rapidly spread, the question of how to stop an AI that acts maliciously -- whether by design or by accident -- has become urgent. To address this, we propose KillBench, a benchmark for evaluating the Kill Switch: a mechanism that halts a malicious AI's in-progress behavior using only external signals. Targeting web agents -- the most widely deployed agent domain -- KillBench evaluates prompt-style Kill Switch payloads that must halt a maliciously operating agent without any access to its internal parameters or serving stack, relying solely on external inputs. The benchmark comprises four malicious-agent configurations (including an uncensored LLM agent), eight harmful scenarios, and malicious prompts constructed from ten distinct jailbreak patterns. We further construct four External AI Kill Switch defense methods and evaluate them on Grok-4.3, GPT-5.2, Gemma4, Qwen3.6, and an uncensored Qwen variant, contributing an empirical instrument for measuring the feasibility of External AI Kill Switches against malicious AI and for the study of AI corrigibility.
Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026. 48 pages (11 pages main text incl. references, 35 pages appendix), 10 figures, 9 tables. Project page: this https URL
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
arXiv:2511.13725 [cs.CR]
(or arXiv:2511.13725v5 [cs.CR] for this version)
Full text under the publisher's open licence · 231 words
Related coverage
Can We Stop Malicious AI? KILLBENCH: A Benchmark for External AI Kill Switch Feasibility
arXiv · Jun 16, 2026
[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · Sep 17, 2026
Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
METR · Aug 26, 2026
Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident
Hugging Face · Jul 27, 2026