aiqbench analysis
This research identifies critical consistency anomalies at the boundary between AI agents and external tools, revealing that current shared interfaces like MCP lack the semantic granularity needed to guarantee transactional integrity during partial failures. By cataloging eight recurring state inconsistencies, the study demonstrates that black-box tool invocation is insufficient for complex workflows and calls for reusable transactional contracts to ensure reliable agent behavior.
When Tool Calls Succeed but Workflows Fail: Anomalies at the Agent-Tool Boundary
arXiv:2609.15397v1 Announce Type: new Abstract: AI agents increasingly execute long-running workflows that externalize effects through independently supplied tools. Under retries, speculative execution, concurrency, and partial failures, the resulting external state may be inconsistent with the workflow's intended resolution: required effects may be missing or duplicated, aborted effects may survive, and committed effects may depend on provisional state that is later withdrawn. Advanced transaction models address related failures, but assume that lower-level operations expose the semantics they depend on: whether an effect occurred, whether it can be compensated, staged, or safely reordered. Shared agent-tool interfaces usually do not. We contribute an effect-history model that separates events in the external world from the runtime's observations of them, and a catalog of eight recurring external-effect anomalies. From the catalog we derive the boundary capabilities required to exclude each anomaly in general, and four points where black-box tool invocation alone cannot provide a general guarantee. We then ask how much of this is expressible in a widely used shared tool interface, measuring the use of the standard annotation vocabulary across 98,291 tools exposed by registered Model Context Protocol (MCP) servers. The fields are widely emitted but provide only coarse call-level hints, and none of the required capabilities is fully expressible. These results motivate reusable transactional contracts at the tool boundary.
Abstract:AI agents increasingly execute long-running workflows that externalize effects through independently supplied tools. Under retries, speculative execution, concurrency, and partial failures, the resulting external state may be inconsistent with the workflow's intended resolution: required effects may be missing or duplicated, aborted effects may survive, and committed effects may depend on provisional state that is later withdrawn. Advanced transaction models address related failures, but assume that lower-level operations expose the semantics they depend on: whether an effect occurred, whether it can be compensated, staged, or safely reordered. Shared agent-tool interfaces usually do not. We contribute an effect-history model that separates events in the external world from the runtime's observations of them, and a catalog of eight recurring external-effect anomalies. From the catalog we derive the boundary capabilities required to exclude each anomaly in general, and four points where black-box tool invocation alone cannot provide a general guarantee. We then ask how much of this is expressible in a widely used shared tool interface, measuring the use of the standard annotation vocabulary across 98,291 tools exposed by registered Model Context Protocol (MCP) servers. The fields are widely emitted but provide only coarse call-level hints, and none of the required capabilities is fully expressible. These results motivate reusable transactional contracts at the tool boundary.
11 pages, 4 tables. Artifact: this https URL
Artificial Intelligence (cs.AI); Databases (cs.DB); Distributed, Parallel, and Cluster Computing (cs.DC); Software Engineering (cs.SE)
arXiv:2609.15397 [cs.AI]
(or arXiv:2609.15397v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.15397 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 324 words
Related coverage

Presentation: Designing AI Platforms for Reliability: Tools for Certainty, Agents for Discovery
InfoQ · May 27, 2026

What are Agentic Workflows?
Databricks · Aug 6, 2026

Friendly Fire: Hijacking Defensive Cyber AI Agents for Remote Code Execution
AI Now Institute · Jul 8, 2026

[AINews] not much happened today
Latent Space · Oct 3, 2026