analiza aiqbench
The paper argues that LLM agents need deterministic code guards for exactly stated limits, paired with learned rules for judgment calls that code can't express. Its results are concrete: a hard guard blocked all unauthorized actions in the tested purchasing, delegation, and data-access episodes, while unguarded models breached often. The authors are careful to say the taught small model matches a prompted frontier model rather than beating it, so the practical takeaway is that a cheap, taught rule layer can work alongside exact guards. Readers building agents should weigh these as results from specific, author-defined domains rather than general guarantees.
Dependable Agents Need Hard and Fuzzy Rules
arXiv:2611.01043v1 Announce Type: cross Abstract: Language models let software read requests in ordinary language, but they keep rules as people do: usually, and less so under pressure. Conventional code keeps rules exactly, but it cannot read a request. We argue that dependable agents need both, because the two kinds of rule cover overlapping and unequal regions of what an agent might do. Code holds the limits that can be stated exactly. Learned fuzzy rules cover the judgment that cannot. We test the hard layer in three domains, purchasing, delegation of authority and data access, on 1,340 episodes written by authors outside our team. Models left to decide for themselves executed unauthorized actions in up to 151 of 340 purchasing episodes and 269 of 470 delegation episodes, and the strongest model breached in every condition. A hard guard between proposal and action executed none, and an exhaustive search of 183,204 proposals agreed. In a further test, where an outsider wrote the owner's rules and a program mapped them to the guard, two episodes breached at a boundary we had written down in advance. We then test the pair. Concept Programming instills a named rule in a model and measures its errors on wording it never saw. A small model taught four judgment rules this way flagged 95 of 96 rule-breaking purchases that no exact rule can see, and 4 of 108 legitimate requests. Placed in front of the guard, it left 1 of 192 rule-breaking requests unstopped, where the guard alone left 77 and 70, and the guard still held every exact limit. A prompted frontier model did about as well, so the claim is that a small taught rule can do the job, not that it does it better.
Abstract:Language models let software read requests in ordinary language, but they keep rules as people do: usually, and less so under pressure. Conventional code keeps rules exactly, but it cannot read a request. We argue that dependable agents need both, because the two kinds of rule cover overlapping and unequal regions of what an agent might do. Code holds the limits that can be stated exactly. Learned fuzzy rules cover the judgment that cannot. We test the hard layer in three domains, purchasing, delegation of authority and data access, on 1,340 episodes written by authors outside our team. Models left to decide for themselves executed unauthorized actions in up to 151 of 340 purchasing episodes and 269 of 470 delegation episodes, and the strongest model breached in every condition. A hard guard between proposal and action executed none, and an exhaustive search of 183,204 proposals agreed. In a further test, where an outsider wrote the owner's rules and a program mapped them to the guard, two episodes breached at a boundary we had written down in advance. We then test the pair. Concept Programming instills a named rule in a model and measures its errors on wording it never saw. A small model taught four judgment rules this way flagged 95 of 96 rule-breaking purchases that no exact rule can see, and 4 of 108 legitimate requests. Placed in front of the guard, it left 1 of 192 rule-breaking requests unstopped, where the guard alone left 77 and 70, and the guard still held every exact limit. A prompted frontier model did about as well, so the claim is that a small taught rule can do the job, not that it does it better.
Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
I.2.11; D.4.6; I.2.6; D.2.4; I.2.7
arXiv:2611.01043 [cs.SE]
(or arXiv:2611.01043v1 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2611.01043 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 386 cuvinte
Subiecte conexe
AsynCodeBench: Benchmarking Collaboration of Asynchronous Multi-Agent Systems in Software Engineering
arXiv · 29 sept. 2026
Multi-Agent Flow Matching with Decoupled Generative Guidance
arXiv · 30 sept. 2026
Code That Works, Environments That Don't: Measuring Environment Reproducibility in AI-Generated Software
arXiv · 2 oct. 2026
Goal-Aware Dynamic Multi-Agent Collaboration for Long-Horizon Tasks
arXiv · 31 oct. 2026