aiqbench analysis
This paper introduces Token Time Continuous Diffusion (TTCD), a novel language modeling approach that operates in continuous space to eliminate the inaccuracies associated with parallel sampling in discrete models. By incorporating per-token time dynamics, TTCD achieves superior performance in conditional generation and high-speed inference while maintaining competitive unconditional quality after self-distillation. The method represents a significant architectural shift toward deterministic diffusion processes for next-token prediction tasks.
Token Time Continuous Diffusion for Language Modeling
arXiv:2607.14106v2 Announce Type: replace Abstract: In this paper we introduce token time continuous diffusion (TTCD), a new diffusion language model which (a) operates in continuous space, deterministically mapping Gaussian noise to a final token canvas with no further sampling, and crucially (b) incorporates a new notion of per-token times, with some tokens proceeding from noise to token at a faster rate than others. Continuous space modeling helps TTCD avoid the parallel sampling of multiple tokens, which is a key source of inaccuracy at high speedups for models that iterate purely in discrete space. The notion of per-token times helps TTCD to better model conditional generation, allows for more sure tokens to proceed at a faster rate, and allows for differentiated inter-token influences during refinement. TTCD outperforms discrete models at high speedups. We train a 160M parameter TTCD model on OpenWebText, and then self-distill it; we find that at high speedups we are comparable in unconditional generation quality, and outperform in conditional generation, several existing models of similar size trained, on the same data, and self-distilled. We achieve similar gains in Sudoku solving as well.
Abstract:In this paper we introduce token time continuous diffusion (TTCD), a new diffusion language model which (a) operates in continuous space, deterministically mapping Gaussian noise to a final token canvas with no further sampling, and crucially (b) incorporates a new notion of per-token times, with some tokens proceeding from noise to token at a faster rate than others. Continuous space modeling helps TTCD avoid the parallel sampling of multiple tokens, which is a key source of inaccuracy at high speedups for models that iterate purely in discrete space. The notion of per-token times helps TTCD to better model conditional generation, allows for more sure tokens to proceed at a faster rate, and allows for differentiated inter-token influences during refinement. TTCD outperforms discrete models at high speedups. We train a 160M parameter TTCD model on OpenWebText, and then self-distill it; we find that at high speedups we are comparable in unconditional generation quality, and outperform in conditional generation, several existing models of similar size trained, on the same data, and self-distilled. We achieve similar gains in Sudoku solving as well.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2607.14106 [cs.CL]
(or arXiv:2607.14106v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2607.14106 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 275 words
Related coverage
Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion
arXiv · Oct 30, 2026
Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling
arXiv · Oct 7, 2026
Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models
arXiv · Oct 2, 2026
A Dominant Self-Conditioning Direction Drives Repetition in Unconditional Continuous Diffusion Language Models
arXiv · Oct 1, 2026