aiqbench analysis
This research introduces the Safety Asymmetry Score (SAS) to quantify how agentic language models differentially process identical malicious payloads based on their arrival channel, such as user messages versus tool metadata. The study reveals a critical security gap where general-purpose models heavily discount instructions in tool metadata compared to agent-native architectures, fundamentally altering the attack surface for autonomous systems. These findings highlight non-linear encoding patterns at mid-to-late model depths that standard linear probes fail to detect, necessitating new evaluation metrics for securing tool-using agents.
Same Payload, Different Channel: Measuring Trust Asymmetry in Tool-Using Language Models
arXiv:2606.00566v2 Announce Type: replace-cross Abstract: As language models take on agentic roles that call APIs, read tool outputs, and act on third-party content, their attack surface expands beyond what users type. Whether they treat a malicious instruction the same way regardless of where it arrives has not been studied systematically. We introduce the Safety Asymmetry Score (SAS), measuring how a model's susceptibility to adversarial content shifts depending on whether it arrives in the user message, tool metadata, or tool output, using matched payload pairs that hold the malicious text identical and vary only the channel. Across 10 production LLMs and three attack families, general-purpose models sharply discount instructions arriving as tool metadata relative to identical instructions in the user message, while agent-native models discount them far less. This differential survives an affordance-matched control equalizing tool availability and scoring, and a size-controlled mixed-effects analysis. Within the tool surface the pattern reverses: the same content is far more instruction-like in a tool's description than in its output, with affordances identical. Models treat tool metadata as instructions and tool results as data. In Llama 3.3-70B and GPT-OSS-120B this signal is causally present at mid-to-late depths but non-linearly encoded, so linear probes miss what activation patching recovers.
Abstract:As language models take on agentic roles that call APIs, read tool outputs, and act on third-party content, their attack surface expands beyond what users type. Whether they treat a malicious instruction the same way regardless of where it arrives has not been studied systematically. We introduce the Safety Asymmetry Score (SAS), measuring how a model's susceptibility to adversarial content shifts depending on whether it arrives in the user message, tool metadata, or tool output, using matched payload pairs that hold the malicious text identical and vary only the channel. Across 10 production LLMs and three attack families, general-purpose models sharply discount instructions arriving as tool metadata relative to identical instructions in the user message, while agent-native models discount them far less. This differential survives an affordance-matched control equalizing tool availability and scoring, and a size-controlled mixed-effects analysis. Within the tool surface the pattern reverses: the same content is far more instruction-like in a tool's description than in its output, with affordances identical. Models treat tool metadata as instructions and tool results as data. In Llama 3.3-70B and GPT-OSS-120B this signal is causally present at mid-to-late depths but non-linearly encoded, so linear probes miss what activation patching recovers.
Machine Learning (cs.LG); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
arXiv:2606.00566 [cs.LG]
(or arXiv:2606.00566v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.00566 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 298 words
Related coverage
Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident
Hugging Face · Jul 27, 2026
More details on Fable 5’s cyber safeguards and our jailbreak framework
Anthropic · Jul 2, 2026
In open RLVR, “improvement” depends on the instrument — a small GRPO testbed separating what training optimizes, measures, and teaches
LessWrong · Jun 15, 2026
Informe de riesgos de la IA de frontera (febrero–marzo de 2026)
METR · May 19, 2026