analiza aiqbench
This arXiv paper is squarely about AI: it studies why discrete diffusion language models may plan better than autoregressive ones, using graph-traversal tasks as a controlled testbed and mechanistic analysis of training and inference. The claims are narrow and based on a minimal setup, so they should be read as insight into model behavior rather than evidence of broad planning superiority. For ML practitioners, the useful takeaway is that architecture choice may shape the internal solution strategy a model learns, which bears on how planning limits in autoregressive models are diagnosed.
Discrete Diffusion Models Exploit Asymmetry to Solve Lookahead Planning Tasks
arXiv:2602.19980v2 Announce Type: replace Abstract: Discrete diffusion models are often said to outperform autoregressive models on planning tasks, but the specific mechanisms enabling this are not yet well understood. To explain why, we abstract the complexity of studying "planning" into graph traversal problems, which offer a minimal setup for evaluating language models' planning and lookahead capabilities. We find that non-autoregressive language models are able to leverage an inherent directional asymmetry in lookahead planning tasks to outperform the training efficiency of their autoregressive counterparts. Through a mechanistic study of the training and inference dynamics of autoregressive and non-autoregressive models, we discover that while forward traversal through junctions in the graph requires complex sequential planning, the reverse path remains deterministic. Consequently, non-autoregressive models naturally develop a fundamentally different solution strategy. Effectively, they learn a reverse-decoding pattern which reduces an $\ell^{th}$-order lookahead problem to a sequence of simple $1^{st}$-order transitions. Latent-space analysis further confirms these divergent strategies by demonstrating the distinct internal representations formed during generation. Ultimately, these findings highlight the advantages of non-autoregressive modeling while clarifying the inherent planning limitations induced by autoregressive architectures.
Abstract:Discrete diffusion models are often said to outperform autoregressive models on planning tasks, but the specific mechanisms enabling this are not yet well understood. To explain why, we abstract the complexity of studying "planning" into graph traversal problems, which offer a minimal setup for evaluating language models' planning and lookahead capabilities. We find that non-autoregressive language models are able to leverage an inherent directional asymmetry in lookahead planning tasks to outperform the training efficiency of their autoregressive counterparts. Through a mechanistic study of the training and inference dynamics of autoregressive and non-autoregressive models, we discover that while forward traversal through junctions in the graph requires complex sequential planning, the reverse path remains deterministic. Consequently, non-autoregressive models naturally develop a fundamentally different solution strategy. Effectively, they learn a reverse-decoding pattern which reduces an $\ell^{th}$-order lookahead problem to a sequence of simple $1^{st}$-order transitions. Latent-space analysis further confirms these divergent strategies by demonstrating the distinct internal representations formed during generation. Ultimately, these findings highlight the advantages of non-autoregressive modeling while clarifying the inherent planning limitations induced by autoregressive architectures.
Machine Learning (cs.LG)
arXiv:2602.19980 [cs.LG]
(or arXiv:2602.19980v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2602.19980 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 269 cuvinte
Subiecte conexe
Differencing the Diffusion Trajectory toward Uncertain Components for Time Series Forecasting
arXiv · 28 iul. 2026
Can Deep Learning Achieve Cross-Physics Mapping?
arXiv · 16 sept. 2026
Membership Inference in Fine-tuned Diffusion Language Models via Token-level Memorization Asymmetry
arXiv · 2 sept. 2026
Schr\"odinger-Bass Bridge Barycenter Models for Measure-Valued Data
arXiv · 22 oct. 2026