analiza aiqbench
This research identifies a critical paradox in sparse-attention decoding where information loss forces models to generate longer sequences, effectively negating efficiency gains. The authors propose an early-stopping mechanism that halts generation once the cost of additional tokens outweighs accuracy benefits, achieving up to 90% token reduction with minimal performance impact on reasoning tasks.
Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage
arXiv:2601.03043v4 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate strong capabilities across a wide range of complex tasks and are increasingly deployed at scale, placing significant demands on inference efficiency. Prior work typically decomposes inference into prefill and decode stages, with the decode stage dominating total latency. To reduce time and memory complexity in the decode stage, a line of work introduces sparse-attention algorithms. In this paper, we show, both empirically and theoretically, that sparse attention can paradoxically increase end-to-end complexity: information loss often induces significantly longer sequences, a phenomenon we term ``Less is Less'' (Lil). To mitigate the Lil problem, we propose an early-stopping algorithm that detects the threshold where information loss exceeds information gain during sparse decoding. Our early-stopping algorithm reduces token consumption by up to 90% with a marginal accuracy degradation of less than 2% across reasoning-intensive benchmarks.
Abstract:Large language models (LLMs) demonstrate strong capabilities across a wide range of complex tasks and are increasingly deployed at scale, placing significant demands on inference efficiency. Prior work typically decomposes inference into prefill and decode stages, with the decode stage dominating total latency. To reduce time and memory complexity in the decode stage, a line of work introduces sparse-attention algorithms. In this paper, we show, both empirically and theoretically, that sparse attention can paradoxically increase end-to-end complexity: information loss often induces significantly longer sequences, a phenomenon we term ``Less is Less'' (Lil). To mitigate the Lil problem, we propose an early-stopping algorithm that detects the threshold where information loss exceeds information gain during sparse decoding. Our early-stopping algorithm reduces token consumption by up to 90% with a marginal accuracy degradation of less than 2% across reasoning-intensive benchmarks.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2601.03043 [cs.CL]
(or arXiv:2601.03043v4 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 155 cuvinte
Subiecte conexe
[AINews] Opus 5.5 is good at explainer videos
Latent Space · 29 sept. 2026
[AINews] The Future of Latent Space
Latent Space · 25 sept. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026
Frontier post-training recipe review with Finbarr Timbers
Interconnects · 16 iun. 2026