analiza aiqbench
This research addresses a critical bottleneck in LLM-based multi-agent systems by introducing EMFA, a method that explicitly models how errors cascade through agent interactions to pinpoint the decisive failure point. By distinguishing root causes from downstream symptoms using counterfactual verification, the approach significantly improves diagnostic accuracy for complex, coordinated reasoning tasks. This work advances the reliability and debuggability of autonomous agent ecosystems, which is essential for deploying them in high-stakes environments.
Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems
arXiv:2610.11600v1 Announce Type: new Abstract: LLM-based multi-agent systems (MASs) are increasingly used to solve complex tasks through coordinated reasoning, tool use, and interaction with external resources. However, attributing failures in such systems remains challenging because the observed outcome often does not directly reveal the error responsible for the failed execution. In this work, the attribution target is the decisive error, defined as the agent--step pair whose correction would recover the failed execution. Existing approaches largely identify suspicious steps without explicitly modeling how errors propagate across interactions or persist in unresolved loops, making decisive errors difficult to distinguish from downstream failure symptoms. We propose \textbf{E}rror-Propagation \textbf{M}odeling for \textbf{F}ailure \textbf{A}ttribution (\textbf{EMFA}). EMFA constructs a structured representation of the failed trajectory, models both cascading propagation and persistent interaction loops, and uses propagation-aware candidate screening followed by counterfactual verification to identify the decisive agent--step pair. On the Who\&When benchmark, EMFA achieves state-of-the-art step-level attribution accuracy and remains competitive at the agent level. It improves the previous best step-level results by 3.45 and 4.40 percentage points on the Hand-Crafted and Algorithm-Generated subsets, respectively.
Abstract:LLM-based multi-agent systems (MASs) are increasingly used to solve complex tasks through coordinated reasoning, tool use, and interaction with external resources. However, attributing failures in such systems remains challenging because the observed outcome often does not directly reveal the error responsible for the failed execution. In this work, the attribution target is the decisive error, defined as the agent--step pair whose correction would recover the failed execution. Existing approaches largely identify suspicious steps without explicitly modeling how errors propagate across interactions or persist in unresolved loops, making decisive errors difficult to distinguish from downstream failure symptoms. We propose \textbf{E}rror-Propagation \textbf{M}odeling for \textbf{F}ailure \textbf{A}ttribution (\textbf{EMFA}). EMFA constructs a structured representation of the failed trajectory, models both cascading propagation and persistent interaction loops, and uses propagation-aware candidate screening followed by counterfactual verification to identify the decisive agent--step pair. On the Who\&When benchmark, EMFA achieves state-of-the-art step-level attribution accuracy and remains competitive at the agent level. It improves the previous best step-level results by 3.45 and 4.40 percentage points on the Hand-Crafted and Algorithm-Generated subsets, respectively.
Artificial Intelligence (cs.AI)
arXiv:2610.11600 [cs.AI]
(or arXiv:2610.11600v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.11600 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 267 cuvinte
Subiecte conexe
Agent Evaluation Metric for multi-turn conversations
AWS · 10 sept. 2026
AgentOps: Operationalize agentic AI at scale with Amazon Bedrock AgentCore
AWS · 1 iun. 2026
StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems
arXiv · 3 iun. 2026
Article: The Platform Engineering Playbook for Production LLMs
InfoQ · 5 oct. 2026