analiza aiqbench
GPT-Red represents a significant advancement in automated adversarial training for large language models by leveraging self-play to discover novel prompt injection attacks. This approach scales safety evaluation using compute resources comparable to major reinforcement learning runs, effectively outperforming human red-teamers and generalizing across diverse defensive environments. The work establishes a critical feedback loop where improved model robustness generates stronger attack signals, driving continuous improvement in frontier AI system security.
GPT-Red: Automated Red Teaming via Self-Play at Scale
arXiv:2607.26115v1 Announce Type: cross Abstract: We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems. To this end, we use it to adversarially train GPT-5.6, our most robust model to prompt injections to date. To create GPT-Red, we design a scalable self-play algorithm where the model is tasked with attacking a diverse population of simultaneously-trained defender agents. We train the model on realistic red-teaming environments using compute on the same scale as some of our largest RL post-training runs, making it the single-largest LLM safety training run ever documented. GPT-Red excels at red-teaming: it reliably breaks our past models up to GPT-5.5, it finds more successful attacks than human red-teamers, and it generalizes to held-out environments, defender models, and harnesses. In the future, we expect that as we improve the robustness of each new GPT model, it will in turn will provide better learning signal for \textit{even stronger} red-teamer agents, thus unlocking a self-improvement flywheel.
Authors:Eric Wallace, Christopher A. Choquette-Choo, Nikhil Kandpal, Sam Toyer, Dylan Hunn, Stephanie Lin, Yuxin Wen, Xiangyu Qi, Christopher Wolff, Zizhao Wang, Milad Nasr, Sicheng Zhu, Chuan Guo, Juan Felipe Cerón Uribe, Kaiwen Wang, Aiden Low, Kai Xiao, Kai Chen
Abstract:We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems. To this end, we use it to adversarially train GPT-5.6, our most robust model to prompt injections to date. To create GPT-Red, we design a scalable self-play algorithm where the model is tasked with attacking a diverse population of simultaneously-trained defender agents. We train the model on realistic red-teaming environments using compute on the same scale as some of our largest RL post-training runs, making it the single-largest LLM safety training run ever documented. GPT-Red excels at red-teaming: it reliably breaks our past models up to GPT-5.5, it finds more successful attacks than human red-teamers, and it generalizes to held-out environments, defender models, and harnesses. In the future, we expect that as we improve the robustness of each new GPT model, it will in turn will provide better learning signal for \textit{even stronger} red-teamer agents, thus unlocking a self-improvement flywheel.
28 pages.13 main pages and 13 main figures
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2607.26115 [cs.CR]
(or arXiv:2607.26115v1 [cs.CR] for this version)
From: Christopher A. Choquette-Choo [view email] [v1] Tue, 28 Jul 2026 16:03:39 UTC (5,979 KB)
Text integral sub licența deschisă a publicației · 259 cuvinte
Subiecte conexe
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · 14 sept. 2026
Digital States 2026: AI Moves From Experiment to Infrastructure
govtech.com · 24 sept. 2026
AXRP Episode 50 - Eli Lifland on AI 2027
LessWrong · 3 aug. 2026
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · 18 sept. 2026