aiqbench analysis
This research advances discrete generative modeling by leveraging insights from random constraint satisfaction problems, demonstrating that continuous diffusion processes outperform masked discrete variants on synthetic benchmarks like k-SAT. The study reveals counterintuitive findings regarding variable ordering and theoretical accuracy limits, offering critical guidance for improving text, tabular, and genomic data generation techniques.
Generating from Discrete Distributions Using Diffusions: Insights from Random Constraint Satisfaction Problems
arXiv:2603.20589v2 Announce Type: replace Abstract: Generating data from discrete distributions is important for a number of application domains including text, tabular data, and genomic data. Several groups have recently used random $k$-satisfiability ($k$-SAT) as a synthetic benchmark for new generative techniques. In this paper, we show that fundamental insights from the theory of random constraint satisfaction problems have observable implications (sometime contradicting intuition) on the behavior of generative techniques on such benchmarks. More precisely, we study the problem of generating a uniformly random solution of a given (random) $k$-SAT or $k$-XORSAT formula. Among other findings, we observe that: $(i)$~Continuous diffusions outperform masked discrete diffusions; $(ii)$~Learned diffusions can match the theoretical `ideal' accuracy; $(iii)$~Smart ordering of the variables can significantly improve accuracy, although not following popular heuristics.
Abstract:Generating data from discrete distributions is important for a number of application domains including text, tabular data, and genomic data. Several groups have recently used random $k$-satisfiability ($k$-SAT) as a synthetic benchmark for new generative techniques. In this paper, we show that fundamental insights from the theory of random constraint satisfaction problems have observable implications (sometime contradicting intuition) on the behavior of generative techniques on such benchmarks. More precisely, we study the problem of generating a uniformly random solution of a given (random) $k$-SAT or $k$-XORSAT formula. Among other findings, we observe that: $(i)$~Continuous diffusions outperform masked discrete diffusions; $(ii)$~Learned diffusions can match the theoretical `ideal' accuracy; $(iii)$~Smart ordering of the variables can significantly improve accuracy, although not following popular heuristics.
Machine Learning (cs.LG)
arXiv:2603.20589 [cs.LG]
(or arXiv:2603.20589v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2603.20589 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 213 words
Related coverage
From Uniform to Learned Graph Priors: Diffusion for Structure Discovery
arXiv · Jun 12, 2026
Continuous Diffusion Scales Competitively with Discrete Diffusion for Language
arXiv · Sep 12, 2026
Scaling Categorical Flow Maps
Apple · Aug 7, 2026
Triton-Based Optimization of Video Sparse Attention on ROCm
AMD · Jul 13, 2026