MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided Rereading

Abstract:Long-context reasoning tasks have motivated agent-memory approaches that process document chunks sequentially while maintaining a dynamically updated memory, avoiding the quadratic complexity of standard attention. Existing methods often rely on retrieval to recover information discarded during memory overwriting, but remain vulnerable to evidence loss during memory formation and interference from invalid queries. To overcome these limitations, we propose MemReread. Built upon streaming reading, it circumvents intermediate retrieval, triggering question decomposition and targeted rereading when the final memory is insufficient. By recovering indirect evidence that may have been prematurely discarded, MemReread enables non-linear reasoning while preserving the logical flow of document comprehension. We further introduce a reinforcement learning framework that improves length extrapolation and adaptively determines the number of rereading passes according to task complexity, enabling flexible control of computational overhead. Our MemReread-4B outperforms strong baselines on long-context reasoning tasks (e.g., +1.6pp v.s. DeepSeek-V4-Pro on 2WikiMultiHopQA) while maintaining linear time complexity with respect to context length.

Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

arXiv:2605.10268 [cs.CL]

(or arXiv:2605.10268v2 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2605.10268 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.