aiqbench analysis
This paper introduces Coca-Cola, a method that compresses tool outputs into continuous latent tokens rather than discrete text, significantly reducing context overhead in LLM agents without sacrificing performance. By employing Action-Preserving Training to align decision distributions, the approach cuts token consumption by up to 89% and runtime by over half while improving task accuracy on web-search benchmarks. This work represents a meaningful advancement in optimizing agent efficiency through novel representation learning techniques.
Tools Need Not Speak in Text: Latent Context Compression for Agents
arXiv:2611.06500v1 Announce Type: new Abstract: Large language model (LLM) agents typically append raw tool outputs to their context. Over multi-step interactions, long and noisy observations incur substantial context overhead and dilute the agent's attention to task-relevant information. Existing token pruning and summarization methods remain constrained by discrete text representations, limiting the achievable compression ratios. In this work, we challenge the prevailing assumption that agents must consume tool observations as explicit text tokens. To this end, we propose Coca-Cola, a continuous latent context compression model for LLM agents. Coca-Cola compresses tool observations into latent tokens and embeds them into the input embedding space of the agent. We further introduce Action-Preserving Training, which aligns the agent's next-action distributions under full and compressed observations. This trains the compression model to learn action-sufficient latent tokens that retain decision-critical information. Experiments with web-search agents on four QA benchmarks and two model families show that Coca-Cola reduces the agent's token consumption by 70.01%-89.24% and end-to-end runtime by 24.45%-55.97%, while consistently improving task performance. These results suggest that Coca-Cola offers a promising approach for context compression in agents. Our code is available at https://github.com/THU-Agent/Coca-cola.
Abstract:Large language model (LLM) agents typically append raw tool outputs to their context. Over multi-step interactions, long and noisy observations incur substantial context overhead and dilute the agent's attention to task-relevant information. Existing token pruning and summarization methods remain constrained by discrete text representations, limiting the achievable compression ratios. In this work, we challenge the prevailing assumption that agents must consume tool observations as explicit text tokens. To this end, we propose Coca-Cola, a continuous latent context compression model for LLM agents. Coca-Cola compresses tool observations into latent tokens and embeds them into the input embedding space of the agent. We further introduce Action-Preserving Training, which aligns the agent's next-action distributions under full and compressed observations. This trains the compression model to learn action-sufficient latent tokens that retain decision-critical information. Experiments with web-search agents on four QA benchmarks and two model families show that Coca-Cola reduces the agent's token consumption by 70.01%-89.24% and end-to-end runtime by 24.45%-55.97%, while consistently improving task performance. These results suggest that Coca-Cola offers a promising approach for context compression in agents. Our code is available at this https URL.
Artificial Intelligence (cs.AI)
arXiv:2611.06500 [cs.AI]
(or arXiv:2611.06500v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2611.06500 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 278 words
Related coverage
Serving DeepSeek-V4: why million-token context is an inference systems problem
Together AI · May 11, 2026
Context Management for Deep Agents
LangChain Blog · Oct 13, 2026
Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference
arXiv · Sep 24, 2026
MORSE: Multi-Context Ordering via Reverse Scoring for Evidence-Preserving Compression
arXiv · Sep 24, 2026