aiqbench analysis
Prefilling-dLLM addresses the quadratic recomputation bottleneck in diffusion language models by introducing a training-free framework that disaggregates prefilling from decoding. By caching key-value representations in chunks and utilizing sparse attention, the method reduces complexity to scale only with decode length while achieving up to 28x speedup at long contexts. This approach also mitigates the lost-in-the-middle phenomenon through periodic attention anchors, offering a significant efficiency gain for long-context inference tasks.
Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models
arXiv:2606.10537v1 Announce Type: new Abstract: Diffusion large language models (dLLMs) re-encode the entire prefix at every denoising step, causing recomputation that scales quadratically with context length and becomes prohibitive for long-context scenarios. We propose Prefilling-dLLM, a training-free prefill-decode disaggregation framework for dLLMs that partitions the prefix into N chunks, caches their KV representations once, and selects the top-K most relevant chunks with intra-chunk token sparsity for decoding, showing that sparse prefilling can outperform dense attention while reducing per-step complexity from quadratic in the full sequence length to quadratic only in the decode length. On LongBench and InfiniteBench, Prefilling-dLLM achieves state-of-the-art quality among dLLM acceleration methods, and an attention kernel that parallelizes decoding over the non-contiguously cached chunk KV yields 9.1--28.0x speedup at 8K--32K contexts. We further show that beginning-of-sequence tokens prepended to each chunk act as periodic attention anchors that eliminate the lost-in-the-middle phenomenon. Code is available at https://github.com/menik1126/Prefilling-dLLM.
Abstract:Diffusion large language models (dLLMs) re-encode the entire prefix at every denoising step, causing recomputation that scales quadratically with context length and becomes prohibitive for long-context scenarios. We propose Prefilling-dLLM, a training-free prefill-decode disaggregation framework for dLLMs that partitions the prefix into N chunks, caches their KV representations once, and selects the top-K most relevant chunks with intra-chunk token sparsity for decoding, showing that sparse prefilling can outperform dense attention while reducing per-step complexity from quadratic in the full sequence length to quadratic only in the decode length. On LongBench and InfiniteBench, Prefilling-dLLM achieves state-of-the-art quality among dLLM acceleration methods, and an attention kernel that parallelizes decoding over the non-contiguously cached chunk KV yields 9.1--28.0x speedup at 8K--32K contexts. We further show that beginning-of-sequence tokens prepended to each chunk act as periodic attention anchors that eliminate the lost-in-the-middle phenomenon. Code is available at this https URL.
Computation and Language (cs.CL)
arXiv:2606.10537 [cs.CL]
(or arXiv:2606.10537v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.10537 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 241 words
Related coverage
GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inference
arXiv · Sep 17, 2026
Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models
arXiv · Sep 1, 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · Sep 29, 2026
[AINews] The Future of Latent Space
Latent Space · Sep 25, 2026