aiqbench analysis
This research addresses the latency and compute overhead of external safety guardrails by training lightweight internal probes to detect harmful content within LLaMA-3.1-8B activations. The study demonstrates that these compact models can match the performance of massive external validators while significantly reducing inference costs, offering a practical path toward safer autonomous systems in resource-constrained environments.
Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models
arXiv:2609.19472v1 Announce Type: new Abstract: Autonomous systems increasingly rely on Large Language Models (LLMs) yet the safety infrastructure surrounding these models introduces latency and compute overhead. This limits utility in resource-constrained, time-critical deployments. Existing external guardrail models remain blind to the model's internal workings, creating a fundamental assurance gap. We ask: does the model already know when the content is harmful? We extract activations from LLaMA-3.1-8B and train lightweight MLP classifier probes (12.6M parameters) to detect harmful prompts. Evaluated on WildJailbreak, Beavertails, and AEGIS 2.0, our probes achieve F1 scores of 99%, 83%, and 84%, respectively competitive with 1000x larger guard models while cutting latency and compute costs.
Abstract:Autonomous systems increasingly rely on Large Language Models (LLMs) yet the safety infrastructure surrounding these models introduces latency and compute overhead. This limits utility in resource-constrained, time-critical deployments. Existing external guardrail models remain blind to the model's internal workings, creating a fundamental assurance gap. We ask: does the model already know when the content is harmful? We extract activations from LLaMA-3.1-8B and train lightweight MLP classifier probes (12.6M parameters) to detect harmful prompts. Evaluated on WildJailbreak, Beavertails, and AEGIS 2.0, our probes achieve F1 scores of 99%, 83%, and 84%, respectively competitive with 1000x larger guard models while cutting latency and compute costs.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
arXiv:2609.19472 [cs.AI]
(or arXiv:2609.19472v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.19472 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
IEEE DSN-W 2026, pp. 48-52
https://doi.org/10.1109/DSN-W70714.2026.00027 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 223 words
Related coverage
[AINews] AI Engineer NYC: Gemini 4 Argon, Vals AI Benchmarks
Latent Space · Oct 14, 2026
Latent reasoning architectures would likely undermine CoT, our strongest oversight tool
AI Alignment Forum · Sep 23, 2026
Beyond Routine Compliance: Cunning Data Cultivates Safety Vigilance in Large Language Models
arXiv · Sep 17, 2026
Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals
METR · Sep 27, 2026