analiza aiqbench
This arXiv paper tests whether "tool graph memory," a technique for guiding LLM agents through multi-tool tasks, actually helps. Across nine models and two benchmarks, it finds little significant gain, and most of the benefit appears to come from extra LLM calls rather than the memory's guidance, which is often wrong. For readers building agents, the practical takeaway is to evaluate memory components against compute-matched baselines, and the paper's finding that textual insights match or beat tool-transition memory at lower cost is a useful point to weigh.
Rethinking the Role of Tool Graph Memory in Multi-Turn Tool Use
arXiv:2610.21571v1 Announce Type: cross Abstract: Large language model (LLM) agents solve complex tasks by calling external tools over multiple turns. Recent work has proposed tool graph memory, which stores past experience as tool transitions to guide subsequent selection. However, it remains unclear whether this component consistently improves agents' tool use or undermines it. We evaluate tool graph memory on nine models and two benchmarks, finding that it seldom improves significantly over baselines, and that its gains, where present, cost over twice the LLM calls. The memory's guidance is often incorrect, and random or no guidance retains most of the gain, implying that the benefit stems from extra computation rather than the memory itself. Moreover, even with the correct next tool suggested, many failures remain, such as giving up without a tool call. Storing the same experience instead as textual insights outperforms tool graph memory with almost no extra LLM calls. This suggests that memory serves better as a soft prior than as a prescription of the next tool, leading to a more effective and generalizable solution.
Abstract:Large language model (LLM) agents solve complex tasks by calling external tools over multiple turns. Recent work has proposed tool graph memory, which stores past experience as tool transitions to guide subsequent selection. However, it remains unclear whether this component consistently improves agents' tool use or undermines it. We evaluate tool graph memory on nine models and two benchmarks, finding that it seldom improves significantly over baselines, and that its gains, where present, cost over twice the LLM calls. The memory's guidance is often incorrect, and random or no guidance retains most of the gain, implying that the benefit stems from extra computation rather than the memory itself. Moreover, even with the correct next tool suggested, many failures remain, such as giving up without a tool call. Storing the same experience instead as textual insights outperforms tool graph memory with almost no extra LLM calls. This suggests that memory serves better as a soft prior than as a prescription of the next tool, leading to a more effective and generalizable solution.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2610.21571 [cs.CL]
(or arXiv:2610.21571v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.21571 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 269 cuvinte
Subiecte conexe
LangChain vs CrewAI: Orchestration Compared to OpenRouter-Native Routing
OpenRouter · 2 oct. 2026
The payload: the semantic layer, memory, and the loop that fills it
Neo4j · 10 aug. 2026
Multi-agent social intelligence with Strands Agents and Amazon Bedrock
AWS · 14 iul. 2026
Performance Profiling on AMD GPUs – Part 5: Profiling-Driven Kernel Optimization with an AI Code-Assist Tool
AMD · 16 iul. 2026