analiza aiqbench
BACON addresses the latency bottleneck in Multimodal Large Language Models by refining KV cache compression techniques, specifically recovering critical visual evidence often lost during aggressive reduction. By calibrating observation window attention with last query signals, this method achieves significant speedups without sacrificing reasoning accuracy, offering a practical optimization for deploying vision-language models on constrained hardware.
Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression
arXiv:2606.14782v3 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) achieve strong vision-language reasoning but incur large KV caches and high decoding latency with long visual contexts. Existing compression methods rely on observation window attention for stable token importance estimation, yet this aggregation can dilute sparse critical evidence and discard answer-relevant tokens under aggressive compression. We identify last query attention as a complementary signal for recovering such evidence, though its irrelevant signals may introduce additional noise. We propose BACON, a plug-and-play method that calibrates observation window attention with last query evidence while suppressing noise through intra-layer coherence and inter-layer persistence. Across diverse benchmarks, models, budgets, and compression methods, BACON improves multimodal KV-cache compression by 7.5% on average under the most aggressive budget, with gains up to 30.9%.
Abstract:Multimodal Large Language Models (MLLMs) achieve strong vision-language reasoning but incur large KV caches and high decoding latency with long visual contexts. Existing compression methods rely on observation window attention for stable token importance estimation, yet this aggregation can dilute sparse critical evidence and discard answer-relevant tokens under aggressive compression. We identify last query attention as a complementary signal for recovering such evidence, though its irrelevant signals may introduce additional noise. We propose BACON, a plug-and-play method that calibrates observation window attention with last query evidence while suppressing noise through intra-layer coherence and inter-layer persistence. Across diverse benchmarks, models, budgets, and compression methods, BACON improves multimodal KV-cache compression by 7.5% on average under the most aggressive budget, with gains up to 30.9%.
Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
arXiv:2606.14782 [cs.CV]
(or arXiv:2606.14782v3 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.14782 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 221 cuvinte
Subiecte conexe
Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression
arXiv · 17 iun. 2026
Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression
arXiv · 16 iun. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026
[AINews] not much happened today
Latent Space · 3 oct. 2026