analiza aiqbench
This arXiv preprint compares continuous diffusion language models (LangFlow) with speculative-drafter architectures (DSpark variants) for standalone text generation, with no target-model verification. Under matched training budgets on OpenWebText, LangFlow's iterative full-sequence refinement does better at longer lengths, while the drafter with recurrent memory leads at length 16. The results are limited to small, from-scratch models and the authors' evaluation budgets, so they are useful evidence about architectural tradeoffs rather than a general ranking.
The Generative Gap Between Speculative Drafter and Continuous Diffusion Language Model
arXiv:2610.40847v1 Announce Type: cross Abstract: Continuous diffusion language models (CDLMs) and speculative drafters offer efficient language generation but are typically evaluated differently: CDLMs generate independently through iterative refinement, whereas drafters propose text for target-model verification and correction. We compare their architectures without verification, using LangFlow for CDLMs and adapting DSpark's Markov and recurrent-head architectures for standalone generation. All models are trained from scratch for unconditional, single-block OpenWebText generation at lengths 16, 128, and 1024, without target features or verification, using common preprocessing, backbone dimensions, and training-update budgets. We evaluate generation perplexity (GenPPL) with an external language model and distributional similarity to reference text with MAUVE. With default denoising steps equal to sequence length, LangFlow outperforms both DSpark variants on both metrics at lengths 128 and 1024, while DSpark-RNN leads at 16. A conditional KL analysis characterizes dependencies excluded by the Markov head's previous-token conditioning. Recurrent memory removes this restriction and improves both metrics, but the longer-sequence gap persists. Teacher probability matching further lowers Markov GenPPL without observed MAUVE gains. Increasing LangFlow's denoising steps at fixed weights improves both metrics at lengths 16 and 128, changing the length-128 ordering relative to DSpark-RNN. These results establish LangFlow's longer-sequence advantage under the evaluated budgets and highlight iterative full-sequence refinement as a route to improving standalone generation beyond gains from recurrent head memory.
Abstract:Continuous diffusion language models (CDLMs) and speculative drafters offer efficient language generation but are typically evaluated differently: CDLMs generate independently through iterative refinement, whereas drafters propose text for target-model verification and correction. We compare their architectures without verification, using LangFlow for CDLMs and adapting DSpark's Markov and recurrent-head architectures for standalone generation. All models are trained from scratch for unconditional, single-block OpenWebText generation at lengths 16, 128, and 1024, without target features or verification, using common preprocessing, backbone dimensions, and training-update budgets. We evaluate generation perplexity (GenPPL) with an external language model and distributional similarity to reference text with MAUVE. With default denoising steps equal to sequence length, LangFlow outperforms both DSpark variants on both metrics at lengths 128 and 1024, while DSpark-RNN leads at 16. A conditional KL analysis characterizes dependencies excluded by the Markov head's previous-token conditioning. Recurrent memory removes this restriction and improves both metrics, but the longer-sequence gap persists. Teacher probability matching further lowers Markov GenPPL without observed MAUVE gains. Increasing LangFlow's denoising steps at fixed weights improves both metrics at lengths 16 and 128, changing the length-128 ordering relative to DSpark-RNN. These results establish LangFlow's longer-sequence advantage under the evaluated budgets and highlight iterative full-sequence refinement as a route to improving standalone generation beyond gains from recurrent head memory.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2610.40847 [cs.CL]
(or arXiv:2610.40847v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.40847 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 312 cuvinte
Subiecte conexe
Run AI Models for Edge Application Development with Just a Few Commands
NVIDIA · 30 oct. 2026
SpecPV: Improving Self-Speculative Decoding for Long-Context Generation via Partial Verification
arXiv · 1 sept. 2026
Frontier Reasoning Reaches the Edge: How to Deploy and Optimize Models on NVIDIA Jetson
NVIDIA · 4 sept. 2026
Speed Up LLM Inference with DSpark Speculative Decoding
KDnuggets · 31 aug. 2026