aiqbench analysis
This paper introduces REMORY, a neural memory network that enhances long-horizon agents by generating soft memory tokens to supplement textual summaries within finite context windows. The approach enables models like Qwen3.8 and GLM-5.3 to approximate full-history performance using only 5.2% of input positions while significantly reducing tool usage errors on benchmarks such as BrowseComp and Terminal-Bench 2.1.
REMORY: Learning Residual Memory for Context Compaction
arXiv:2610.11287v1 Announce Type: cross Abstract: Long-horizon agents compact their history to continue within a finite context window, but a textual summary alone may not support every subsequent decision. We introduce REMORY, a neural memory network that supplements the summary with a bounded sequence of soft memory tokens. Given the history and summary, the network learns to generate tokens that help a frozen LLM approximate the continuation it would produce with the full history. The tokens are conditioned on the summary and appended after it, forming an analogue of a residual connection along the sequence dimension. On SummHay, REMORY improves source attribution at nearly unchanged insight coverage and approaches the full-context joint score using only 5.2% of the input positions. Across long-horizon agent benchmarks, Qwen3.8-27B and GLM-5.3-Flash show consistent gains with residual memory. Both models also exhibit substantially fewer repeated tool outputs and tool errors on BrowseComp and Terminal-Bench 2.1.
Abstract:Long-horizon agents compact their history to continue within a finite context window, but a textual summary alone may not support every subsequent decision. We introduce REMORY, a neural memory network that supplements the summary with a bounded sequence of soft memory tokens. Given the history and summary, the network learns to generate tokens that help a frozen LLM approximate the continuation it would produce with the full history. The tokens are conditioned on the summary and appended after it, forming an analogue of a residual connection along the sequence dimension. On SummHay, REMORY improves source attribution at nearly unchanged insight coverage and approaches the full-context joint score using only 5.2% of the input positions. Across long-horizon agent benchmarks, Qwen3.8-27B and GLM-5.3-Flash show consistent gains with residual memory. Both models also exhibit substantially fewer repeated tool outputs and tool errors on BrowseComp and Terminal-Bench 2.1.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2610.11287 [cs.CL]
(or arXiv:2610.11287v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.11287 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 242 words
Related coverage
More Than Memory: Task-Conditioned Signed FFN Writes in Long-Context Retrieval
arXiv · Jul 21, 2026
ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference
arXiv · Oct 1, 2026
Dual-Cache Latent Space Communication between Heterogeneous Language Models
arXiv · Aug 24, 2026
Exact ReLU realization of affine one-dimensional refinement iterates via residual memory and offset frames
arXiv · Jul 24, 2026