analiza aiqbench
SARDI introduces a training-free, retriever-agnostic framework that leverages discarded low-confidence tokens from discrete diffusion models as lookahead signals for retrieval-augmented generation. This approach enables dynamic RAG during the denoising process, delivering up to 8x higher throughput on multi-hop QA benchmarks compared to current autoregressive baselines. The method's applicability to any reasoning-capable discrete diffusion model suggests significant potential for optimizing inference efficiency without additional training costs.
Self-Augmenting Retrieval for Diffusion Language Models
arXiv:2606.06474v1 Announce Type: cross Abstract: Discrete diffusion language models generate text by iteratively denoising an entire response in parallel. At each step, they predict tentative tokens for every masked position, committing the confident predictions to the output and discarding the unconfident ones. We show that the discarded tokens are in fact a useful lookahead signal for retrieval-augmented generation: even low-confidence tokens often surface salient entities early in the denoising trajectory, enabling retrieval of stronger evidence before the output is finalized. We exploit this through Self-Augmenting Retrieval for Diffusion Language Models (SARDI), a dynamic RAG framework that uses these lookahead tokens to guide retrieval during denoising. SARDI is training-free, retriever-agnostic, and applicable to any reasoning-capable discrete diffusion language model. Across five multi-hop QA benchmarks, SARDI outperforms current training-free diffusion and autoregressive retrieval baselines at up to $8\times$ higher throughput.
Abstract:Discrete diffusion language models generate text by iteratively denoising an entire response in parallel. At each step, they predict tentative tokens for every masked position, committing the confident predictions to the output and discarding the unconfident ones. We show that the discarded tokens are in fact a useful lookahead signal for retrieval-augmented generation: even low-confidence tokens often surface salient entities early in the denoising trajectory, enabling retrieval of stronger evidence before the output is finalized. We exploit this through Self-Augmenting Retrieval for Diffusion Language Models (SARDI), a dynamic RAG framework that uses these lookahead tokens to guide retrieval during denoising. SARDI is training-free, retriever-agnostic, and applicable to any reasoning-capable discrete diffusion language model. Across five multi-hop QA benchmarks, SARDI outperforms current training-free diffusion and autoregressive retrieval baselines at up to $8\times$ higher throughput.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2606.06474 [cs.CL]
(or arXiv:2606.06474v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.06474 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 234 cuvinte
Subiecte conexe
LADDER: Graph-Guided Diffusion Language Models for Efficient Multi-Hop Reasoning
arXiv · 22 sept. 2026
[AINews] OpenAI DevDay 2026: Dots, 6.1 Sol, Ultrafast, Decisions API, Agents API, Spaces, Marketplace, and 1.2 Billion ChatGPT WAU
Latent Space · 30 sept. 2026
Introducing Mistral OCR 4
Mistral AI · 23 iun. 2026
Article: Your Next DSL Author is a Language Model
InfoQ · 16 sept. 2026