analiza aiqbench
This paper introduces Simplax, a method that augments discrete diffusion models with continuous simplex variables to improve generative quality without altering the underlying categorical corruption process. The approach yields a tractable reverse sampler and demonstrates superior performance on text generation tasks like OpenWebText and logic puzzles such as Sudoku. By bridging discrete and continuous representations, this work advances the theoretical foundations of categorical generation in machine learning.
Simplex Relaxation for Discrete Diffusion
arXiv:2608.10615v2 Announce Type: replace Abstract: Discrete diffusion models for categorical generation are defined by a corruption kernel, which determines the intermediate state space and the associated reverse prediction problem. Within this family, uniform diffusion has been extensively developed, with recent work connecting its categorical corruption process to continuous representations and dynamics. Motivated by this view, we ask whether uniform discrete diffusion can be augmented with an explicit continuous state while leaving its categorical corruption process unchanged. We introduce Simplax, an exact Dirichlet--categorical augmentation that couples each corrupted categorical state with an auxiliary simplex-valued variable while preserving the uniform diffusion process as its categorical marginal. This augmentation yields a tractable Rao--Blackwellized reverse-bridge objective and a stochastic reverse sampler, while retaining the corrupted categorical state as the denoiser input. Empirically, Simplax improves the generative perplexity--entropy tradeoff on unconditional OpenWebText generation. On Sudoku, a model trained exclusively on $30$-clue puzzles achieves the highest accuracy among the compared methods across all evaluated clue densities, including the minimum uniquely solvable $17$-clue regime, and also achieves the highest validity in unconditional generation.
Abstract:Discrete diffusion models for categorical generation are defined by a corruption kernel, which determines the intermediate state space and the associated reverse prediction problem. Within this family, uniform diffusion has been extensively developed, with recent work connecting its categorical corruption process to continuous representations and dynamics. Motivated by this view, we ask whether uniform discrete diffusion can be augmented with an explicit continuous state while leaving its categorical corruption process unchanged. We introduce Simplax, an exact Dirichlet--categorical augmentation that couples each corrupted categorical state with an auxiliary simplex-valued variable while preserving the uniform diffusion process as its categorical marginal. This augmentation yields a tractable Rao--Blackwellized reverse-bridge objective and a stochastic reverse sampler, while retaining the corrupted categorical state as the denoiser input. Empirically, Simplax improves the generative perplexity--entropy tradeoff on unconditional OpenWebText generation. On Sudoku, a model trained exclusively on $30$-clue puzzles achieves the highest accuracy among the compared methods across all evaluated clue densities, including the minimum uniquely solvable $17$-clue regime, and also achieves the highest validity in unconditional generation.
Computation and Language (cs.CL)
arXiv:2608.10615 [cs.CL]
(or arXiv:2608.10615v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.10615 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 264 cuvinte
Subiecte conexe
Simplex Diffusion Models
arXiv · 30 sept. 2026
Impute-EM: Native Mixed-State Diffusion Models for Heterogeneous Data Imputation
arXiv · 15 sept. 2026
Learning Permutation Distributions via Reflected Diffusion on Ranks
arXiv · 16 iun. 2026
Guided Diffusion by Optimized Loss Functions on Relaxed Parameters for Inverse Material Design
arXiv · 21 aug. 2026