aiqbench analysis
This research establishes rigorous convergence rates for Wasserstein gradient flows driven by interaction kernels relevant to transformer architectures, such as zonal kernels on spheres and regularized Riesz kernels. By proving that energy gaps decay faster than $t^{-1}$ without relying on standard diffusion or geodesic convexity, the study advances the theoretical foundations of optimization dynamics in machine learning models.
Sharp Convergence of Wasserstein Gradient Flows for Spectrally Nonnegative Interaction Energies
arXiv:2609.27008v1 Announce Type: cross Abstract: We study the long-time behavior of Wasserstein gradient flows for interaction energies \[ \mathsf E[\mu] = \frac12\iint_{M\times M}K(x,y)\,\mathrm d\mu(x)\,\mathrm d\mu(y) \] on a closed manifold $M$. For kernels diagonal in a Laplace eigenbasis with nonnegative spectral coefficients, we prove a differential inequality relating the relative entropy to the energy gap. Consequently, for any nonnegative initial density $u_0\in L^p(M)$, $p>1$, the energy gap is integrable in time and satisfies \[ \mathsf E[\mu_t]-\mathsf E_{\min}=o(t^{-1}). \] If all spectral coefficients are positive, the flow converges weakly to the constant measure. These interaction energies need not be geodesically convex in Wasserstein space, and the associated flows contain no diffusion; their global convergence therefore does not follow from standard Wasserstein gradient flow theory. The kernels covered by our results include zonal kernels on spheres, kernels arising in transformer models, regularized Riesz kernels, and inverse fractional Laplacian kernels. We also investigate the sharpness of the $o(t^{-1})$ rate. For any smooth kernel in this class with infinitely many positive spectral coefficients and any $\delta>0$, we construct a solution of the linearized flow whose energy is comparable to $t^{-1-\delta}$ along a sequence of times tending to infinity. Moreover, for any $\delta>0$, by choosing a suitable inverse fractional Laplacian kernel on the flat torus, we construct an exact solution of the nonlinear Wasserstein gradient flow whose energy is comparable to $t^{-1-\delta}$. The nonlinear construction is based on uniform-in-time estimates for the evolution of the dyadic Fourier coefficient blocks and a blockwise energy-persistence argument. These estimates also yield a uniform-in-time quantitative comparison between the nonlinear Wasserstein gradient flow and its linearization.
Abstract:We study the long-time behavior of Wasserstein gradient flows for interaction energies \[ \mathsf E[\mu] = \frac12\iint_{M\times M}K(x,y)\,\mathrm d\mu(x)\,\mathrm d\mu(y) \] on a closed manifold $M$. For kernels diagonal in a Laplace eigenbasis with nonnegative spectral coefficients, we prove a differential inequality relating the relative entropy to the energy gap. Consequently, for any nonnegative initial density $u_0\in L^p(M)$, $p>1$, the energy gap is integrable in time and satisfies \[ \mathsf E[\mu_t]-\mathsf E_{\min}=o(t^{-1}). \] If all spectral coefficients are positive, the flow converges weakly to the constant measure. These interaction energies need not be geodesically convex in Wasserstein space, and the associated flows contain no diffusion; their global convergence therefore does not follow from standard Wasserstein gradient flow theory. The kernels covered by our results include zonal kernels on spheres, kernels arising in transformer models, regularized Riesz kernels, and inverse fractional Laplacian kernels. We also investigate the sharpness of the $o(t^{-1})$ rate. For any smooth kernel in this class with infinitely many positive spectral coefficients and any $\delta>0$, we construct a solution of the linearized flow whose energy is comparable to $t^{-1-\delta}$ along a sequence of times tending to infinity. Moreover, for any $\delta>0$, by choosing a suitable inverse fractional Laplacian kernel on the flat torus, we construct an exact solution of the nonlinear Wasserstein gradient flow whose energy is comparable to $t^{-1-\delta}$. The nonlinear construction is based on uniform-in-time estimates for the evolution of the dyadic Fourier coefficient blocks and a blockwise energy-persistence argument. These estimates also yield a uniform-in-time quantitative comparison between the nonlinear Wasserstein gradient flow and its linearization.
Analysis of PDEs (math.AP); Machine Learning (cs.LG); Optimization and Control (math.OC)
arXiv:2609.27008 [math.AP]
(or arXiv:2609.27008v1 [math.AP] for this version)
https://doi.org/10.48550/arXiv.2609.27008 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 363 words
Related coverage
An operator splitting analysis of Wasserstein--Fisher--Rao gradient flows
arXiv · Sep 17, 2026
GAC-PINN: Geometry-Adaptive and Constraint-Enhanced Physics-Informed Neural Networks
arXiv · Sep 29, 2026
Wasserstein Gradient Flows and Forward-Only Diffusion Are Not Enough for Multimodal Sampling
arXiv · Oct 2, 2026
Opinion Leader Dynamics: How Sparse Attention Shapes Token Clustering
arXiv · Sep 22, 2026