analiza aiqbench
This research addresses a critical bottleneck in speculative decoding by introducing DBLast, which handles token dependencies often ignored by standard block drafters during stochastic sampling. By optimizing the drafter for non-greedy regimes with higher entropy, the method significantly improves accepted sequence lengths without compromising model accuracy on diverse benchmarks like GSM8K and HumanEval. This advancement directly enhances inference efficiency for large language models operating in realistic probabilistic generation scenarios.
DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding
arXiv:2608.05448v1 Announce Type: new Abstract: Speculative decoding accelerates large language models' inference by using a lightweight drafter to propose multiple future tokens and a target model to verify them. While recent block and diffusion-style drafters can predict several positions in a single pass, their training and sampling procedures are typically optimized for greedy decoding or assume that positions in the draft block are conditionally independent. This assumption becomes brittle in non-greedy speculative decoding, where the target distribution is deliberately stochastic and multiple continuations become plausible. We study this mismatch for block diffusion drafters and show that the accepted draft length degrades as the entropy of the target sampling distribution increases. We propose a dependent block drafter based on a low-rank latent mixture over token positions, complemented by an acceptance-oriented training objective that directly targets the expected verified length. Experiments with Qwen3-4B and Qwen3-8B on GSM8K, MT-Bench, HumanEval, and creative-writing benchmarks show that our approach, namely DBLast, consistently improves accepted length over independent block sampling, especially in higher-entropy decoding regimes.
Abstract:Speculative decoding accelerates large language models' inference by using a lightweight drafter to propose multiple future tokens and a target model to verify them. While recent block and diffusion-style drafters can predict several positions in a single pass, their training and sampling procedures are typically optimized for greedy decoding or assume that positions in the draft block are conditionally independent. This assumption becomes brittle in non-greedy speculative decoding, where the target distribution is deliberately stochastic and multiple continuations become plausible. We study this mismatch for block diffusion drafters and show that the accepted draft length degrades as the entropy of the target sampling distribution increases. We propose a dependent block drafter based on a low-rank latent mixture over token positions, complemented by an acceptance-oriented training objective that directly targets the expected verified length. Experiments with Qwen3-4B and Qwen3-8B on GSM8K, MT-Bench, HumanEval, and creative-writing benchmarks show that our approach, namely DBLast, consistently improves accepted length over independent block sampling, especially in higher-entropy decoding regimes.
Computation and Language (cs.CL)
arXiv:2608.05448 [cs.CL]
(or arXiv:2608.05448v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.05448 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 259 cuvinte
Subiecte conexe
Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds
arXiv · 8 oct. 2026
Contextual Correction of Parallel Draftsfor Speculative Decoding
arXiv · 12 oct. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026
DFlash Speculative Decoding on AMD Instinct MI355X: Up to 5× Faster Qwen3.5 Inference
AMD · 16 sept. 2026