aiqbench analysis
This research exposes fundamental limitations in discrete diffusion models, demonstrating that standard confidence-ranking samplers fail to capture inherent token dependencies and generate distributions significantly deviating from the training data. The study proves that per-position marginals are insufficient for modeling joint distributions, revealing a critical gap between per-sample metrics and actual sampling noise floors in generative AI systems.
Limits of Confidence in Diffusion
arXiv:2609.20581v1 Announce Type: new Abstract: Discrete diffusion, including remasking and uniform-state samplers, generate a sequence by writing multiple token positions per step, drawing each from a per-position distribution and choosing which positions to write from those same distributions. For domains of general interest (pixels, phonemes, or words) there are inherent dependencies between tokens. We show that a step matches the training distribution only when the positions it writes are conditionally independent given the tokens already fixed, that no product of per-position distributions can match a dependent group, and that per-position distributions do not determine whether a group is dependent: two joint distributions can have identical per-position marginals while differing in which combinations of values occur. On ScanAndAdd, a synthetic task whose joint distribution is available in closed form, we verify that every group of two or more undetermined positions a confidence ranking writes is dependent, and measure the generated distribution to be $29\times$ the sampling-noise floor total variation while per-sample metrics are $1.0$.
Abstract:Discrete diffusion, including remasking and uniform-state samplers, generate a sequence by writing multiple token positions per step, drawing each from a per-position distribution and choosing which positions to write from those same distributions. For domains of general interest (pixels, phonemes, or words) there are inherent dependencies between tokens. We show that a step matches the training distribution only when the positions it writes are conditionally independent given the tokens already fixed, that no product of per-position distributions can match a dependent group, and that per-position distributions do not determine whether a group is dependent: two joint distributions can have identical per-position marginals while differing in which combinations of values occur. On ScanAndAdd, a synthetic task whose joint distribution is available in closed form, we verify that every group of two or more undetermined positions a confidence ranking writes is dependent, and measure the generated distribution to be $29\times$ the sampling-noise floor total variation while per-sample metrics are $1.0$.
Artificial Intelligence (cs.AI)
arXiv:2609.20581 [cs.AI]
(or arXiv:2609.20581v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.20581 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 252 words
Related coverage
Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs
arXiv · Aug 5, 2026
$x$-Prediction Flow: Efficient Continuous Decoding for Masked Diffusion Language Models
arXiv · Aug 18, 2026
One-step lowest-variance selection in a Gaussian random-field model motivated by masked diffusion: Total correlation and a square root collision threshold
arXiv · Jul 21, 2026
D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models
arXiv · Jun 5, 2026