aiqbench analysis
This research introduces a continuous decoding framework for masked diffusion language models, replacing binary token commitments with reversible partial progress to significantly improve efficiency under limited computational budgets. By integrating confidence-based asynchronous updates and reinforcement learning policies, the method retains high accuracy while using fewer than 15% of standard diffusion steps on pretrained LLaDA architectures.
$x$-Prediction Flow: Efficient Continuous Decoding for Masked Diffusion Language Models
arXiv:2606.29066v2 Announce Type: replace Abstract: Masked diffusion language models (MDLMs) generate text by iteratively unmasking tokens, but their standard decoder reduces each step to a binary action: a position is either committed to a single token or left fully masked, discarding rich predictive information rather than carrying it forward, and forcing premature, irrevocable commitments that lead to poor performance under a limited decoding budget. In this paper, we reinterpret mask prediction as a clean-state prediction ($x$-prediction) and show that it can be used to induce a continuous flow in the input embedding space. Building on this view, we propose a continuous decoding framework for MDLMs where tokens can accumulate partial progress at each diffusion step and remain revisable. To match the uneven contextual constraints across positions in language, we replace the globally synchronous schedule in image diffusion with a confidence-based asynchronous update in which the diffusion progress is token-wise accumulated. Additionally, we introduce a lightweight policy network and formulate its training as a reinforcement learning problem. Applied to pretrained LLaDA, our decoder retains 83--97% of full-budget accuracy using under 15% of the diffusion steps, largely outperforming discrete mask-prediction decoding at matched budgets.
Abstract:Masked diffusion language models (MDLMs) generate text by iteratively unmasking tokens, but their standard decoder reduces each step to a binary action: a position is either committed to a single token or left fully masked, discarding rich predictive information rather than carrying it forward, and forcing premature, irrevocable commitments that lead to poor performance under a limited decoding budget. In this paper, we reinterpret mask prediction as a clean-state prediction ($x$-prediction) and show that it can be used to induce a continuous flow in the input embedding space. Building on this view, we propose a continuous decoding framework for MDLMs where tokens can accumulate partial progress at each diffusion step and remain revisable. To match the uneven contextual constraints across positions in language, we replace the globally synchronous schedule in image diffusion with a confidence-based asynchronous update in which the diffusion progress is token-wise accumulated. Additionally, we introduce a lightweight policy network and formulate its training as a reinforcement learning problem. Applied to pretrained LLaDA, our decoder retains 83--97% of full-budget accuracy using under 15% of the diffusion steps, largely outperforming discrete mask-prediction decoding at matched budgets.
Computation and Language (cs.CL)
arXiv:2606.29066 [cs.CL]
(or arXiv:2606.29066v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.29066 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 280 words
Related coverage
Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
AWS · Sep 4, 2026
One Latent, Many Tokens: Jointly Learning Compressed Embeddings for Efficient Language Diffusion
arXiv · Sep 29, 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026
ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks
arXiv · Sep 26, 2026