analiza aiqbench
TaskPress addresses the linear memory bottleneck in long-context LLM inference by introducing query-agnostic KV cache compression guided by high-level task semantics. By treating a meta-query as a filter during prefill, this method creates reusable memory representations that significantly reduce storage overhead without sacrificing performance across diverse downstream tasks. This advancement represents a critical efficiency gain for deploying large models with extended context windows in resource-constrained environments.
TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning
arXiv:2608.03276v1 Announce Type: new Abstract: Long-context inference with large language models is constrained by the linear growth of the key-value cache to sequence length. While pruning offers mitigation, prevailing methods determine query-specific token importance that cannot be reused across unseen queries. In contrast, we introduce TaskPress, a framework for task-guided, query-agnostic KV cache eviction. Instead of optimizing the cache for a single query, TaskPress constructs a reusable memory representation conditioned on a high-level task guide. The guide functions as a meta-query during prefill to filter irrelevant tokens before downstream queries are issued. In addition, TaskPress leverages quantization scale factors as a zero-cost signal for detecting influential representation outliers, providing an efficient proxy for token importance. Experiments on conducted on various tasks with long context input demonstrate that TaskPress efficiently creates a compact, reusable cache across diverse queries.
Abstract:Long-context inference with large language models is constrained by the linear growth of the key-value cache to sequence length. While pruning offers mitigation, prevailing methods determine query-specific token importance that cannot be reused across unseen queries. In contrast, we introduce TaskPress, a framework for task-guided, query-agnostic KV cache eviction. Instead of optimizing the cache for a single query, TaskPress constructs a reusable memory representation conditioned on a high-level task guide. The guide functions as a meta-query during prefill to filter irrelevant tokens before downstream queries are issued. In addition, TaskPress leverages quantization scale factors as a zero-cost signal for detecting influential representation outliers, providing an efficient proxy for token importance. Experiments on conducted on various tasks with long context input demonstrate that TaskPress efficiently creates a compact, reusable cache across diverse queries.
Artificial Intelligence (cs.AI)
arXiv:2608.03276 [cs.AI]
(or arXiv:2608.03276v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2608.03276 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 226 cuvinte
Subiecte conexe
[AINews] not much happened today
Latent Space · 10 sept. 2026
Task-Aware Structured Memory for Dynamic Multi-modal In-Context Learning
arXiv · 11 iun. 2026
Agentic Systems: A Practitioner's Guide to 6 Advanced Architectural Patterns
Towards Data Science · 11 oct. 2026
EchoPress: Query-Agnostic KV Cache Pruning via Virtual Context Reconstruction
arXiv · 2 oct. 2026