analiza aiqbench
This research introduces TAGGRAPH, a framework for evaluating graph-based retrieval systems designed to enhance the long-term memory of LLM agents by integrating chronological edges and diffusion techniques. The study provides a critical comparative analysis against lexical baselines like BM25, revealing that while graph methods excel in specific contexts, raw input references often achieve higher Mean Reciprocal Rank scores on standard benchmarks. Ultimately, the findings emphasize that retrieval strategy selection must be jointly evaluated with memory extraction quality and store size constraints to ensure robust agent consistency.
TAGGRAPH: Tag-Augmented Graphs for Graph Retrieval of Agent Persistent Histories
arXiv:2609.38353v1 Announce Type: cross Abstract: Long-term memory lets LLM agents recall past interactions and remain consistent across sessions, but memory systems are hard to compare because they often vary in representation, indexing, retrieval, and evaluation. We present a controlled evaluation framework based on shared 5W-style conversational memories. Localized graph configurations traverse a common base graph; AdaptiveGraph adds chronological edges and Personalized PageRank diffusion. We also evaluate BM25 over the same extracted notes and OpenClaw as a raw-input external reference. Retrieval rankings vary across memory settings. On LongMemEval-S, AdaptiveGraph is the strongest graph configuration at 0.844 MRR, but BM25 reaches 0.867 and OpenClaw 0.880. On ATANT Core, localized graph traversal outperforms diffusion and BM25, whereas BM25 leads the stress rounds. Reducing LongMemEval-S within the tested range does not reproduce the ATANT diffusion penalty, but the smallest tested store remains larger than ATANT Core, so store size cannot be ruled out. The penalty also persists under a permissive content-match criterion. Vocabulary normalization and extraction quality substantially affect graph retrieval, and missing extraction tags are common among top-five misses. Retrieval strategies should therefore be evaluated jointly with the memory setting and against strong lexical baselines.
Abstract:Long-term memory lets LLM agents recall past interactions and remain consistent across sessions, but memory systems are hard to compare because they often vary in representation, indexing, retrieval, and evaluation. We present a controlled evaluation framework based on shared 5W-style conversational memories. Localized graph configurations traverse a common base graph; AdaptiveGraph adds chronological edges and Personalized PageRank diffusion. We also evaluate BM25 over the same extracted notes and OpenClaw as a raw-input external reference. Retrieval rankings vary across memory settings. On LongMemEval-S, AdaptiveGraph is the strongest graph configuration at 0.844 MRR, but BM25 reaches 0.867 and OpenClaw 0.880. On ATANT Core, localized graph traversal outperforms diffusion and BM25, whereas BM25 leads the stress rounds. Reducing LongMemEval-S within the tested range does not reproduce the ATANT diffusion penalty, but the smallest tested store remains larger than ATANT Core, so store size cannot be ruled out. The penalty also persists under a permissive content-match criterion. Vocabulary normalization and extraction quality substantially affect graph retrieval, and missing extraction tags are common among top-five misses. Retrieval strategies should therefore be evaluated jointly with the memory setting and against strong lexical baselines.
Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
arXiv:2609.38353 [cs.IR]
(or arXiv:2609.38353v1 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2609.38353 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 284 cuvinte
Subiecte conexe
Your AI Agent Forgets Everything. So We Taught It to Remember — With a Graph.
Neo4j · 7 sept. 2026
A Graph-Native Bitemporal Memory Store for Conversational AI Agents
arXiv · 31 iul. 2026
Accurate and Efficient Long-Term Memory for LLM Agents
arXiv · 21 iul. 2026
Event-Centric Memory with Query-Aware Graph Augmentation for Long-Term Conversational Agents
arXiv · 9 oct. 2026