WIRE: Profiling Witnessed Within-Policy Instruction Collisions in LLM Agents

arXiv:2605.27784v2 Announce Type: replace Abstract: LLM agents are governed by long-lived prompt policies, where individually reasonable stand- ing rules can jointly govern the same pre- generation state. Existing instruction-following evaluations usually ask whether a model satis- fies explicit constraints, but they do not show how a model resolves pressure among rules inside one standing policy. We introduce WIRE, a witnessed resolu- tion profiler for prompt policies. WIRE ex- tracts source-grounded rules, encodes them as PYRULE clauses, uses satisfiability checks only to nominate same-surface hard-collision can- didates, realizes those candidates as concrete co-governance witnesses, and executes subject models to produce a four-cell resolution profile: satisfy both rules, only the earlier rule, only the later rule, or neither. Across six public prompt policies, WIRE ex- tracts 276 source rules and 560 clauses, clas- sifies 30,944 within-policy clause-pair com- parisons, retains 170 encoded hard-collision source-rule pairs, and realizes 1,402 concrete witnesses. In policy-only evaluation, these wit- nesses yield 13,335 jointly governed, judgeable trials; only 35.4% satisfy both governed rules. The resulting profiles reveal policy-specific, model-specific, and tool-interface-specific res- olution patterns. WIRE is not a proof of natural-language contra- diction, a deployment-frequency estimator, or a root-cause diagnosis. It is a measurement tool that returns reproducible witnesses and aggre- gate profiles for inspection, regression testing, and repair.

Abstract:LLM agents are governed by long-lived prompt policies, where individually reasonable stand- ing rules can jointly govern the same pre- generation state. Existing instruction-following evaluations usually ask whether a model satis- fies explicit constraints, but they do not show how a model resolves pressure among rules inside one standing policy. We introduce WIRE, a witnessed resolu- tion profiler for prompt policies. WIRE ex- tracts source-grounded rules, encodes them as PYRULE clauses, uses satisfiability checks only to nominate same-surface hard-collision can- didates, realizes those candidates as concrete co-governance witnesses, and executes subject models to produce a four-cell resolution profile: satisfy both rules, only the earlier rule, only the later rule, or neither. Across six public prompt policies, WIRE ex- tracts 276 source rules and 560 clauses, clas- sifies 30,944 within-policy clause-pair com- parisons, retains 170 encoded hard-collision source-rule pairs, and realizes 1,402 concrete witnesses. In policy-only evaluation, these wit- nesses yield 13,335 jointly governed, judgeable trials; only 35.4% satisfy both governed rules. The resulting profiles reveal policy-specific, model-specific, and tool-interface-specific res- olution patterns. WIRE is not a proof of natural-language contra- diction, a deployment-frequency estimator, or a root-cause diagnosis. It is a measurement tool that returns reproducible witnesses and aggre- gate profiles for inspection, regression testing, and repair.

Artificial Intelligence (cs.AI)

arXiv:2605.27784 [cs.AI]

(or arXiv:2605.27784v2 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2605.27784 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

WIRE: Profiling Witnessed Within-Policy Instruction Collisions in LLM Agents

Related coverage