aiqbench analysis
This research establishes a theoretical lower bound for training loss in diffusion models, identifying the "loss floor" as an intrinsic component derived from Fisher-Rao geometry and Schrödinger bridge principles. By decomposing the denoising objective into information flow and corruption schedule weights, the study explains why raw losses across different noise ranges may inconsistently rank model performance. These findings provide a rigorous geometric framework for understanding training dynamics in high-dimensional latent spaces without relying on empirical heuristics.
The Loss Floor of Denoising Score Matching: Fisher Geometry from Schr\"odinger Bridges
arXiv:2608.23916v1 Announce Type: new Abstract: Denoising score matching trains diffusion models by regressing onto a conditional score, although generation ultimately requires the marginal score. The two objectives share the same population minimizer, but the conditional target remains random at fixed noisy state and introduces an irreducible excess in the training loss. We isolate this excess and show that, for a general corruption kernel under mild regularity assumptions, it is exactly the trace of the Fisher--Rao metric of the conditional endpoint family, integrated along the diffusion trajectory. This gives an exact conditional-variance decomposition of the denoising objective and identifies the information geometry observed in diffusion latent spaces as an intrinsic component of the training loss. We derive the result from a Schr"odinger bridge variational principle, in which the ideal objective arises as excess path-space relative entropy. For corruption diffusions, the Fisher term is proportional to the rate at which the noisy state loses mutual information about the clean data, separating the loss floor into an information flow determined by the data and a weight determined by the corruption schedule and objective. In the Gaussian case, this yields a closed form for the floor, recovers reparametrization invariance of the continuous-time objective, and relates its high-SNR divergence to the information dimension of the data. Finally, we show that raw losses obtained with different noise ranges or weightings need not rank models consistently because they contain different additive floors, and contrast the second-order geometry seen by training with the third-order conditional statistics entering numerical sampling error.
Abstract:Denoising score matching trains diffusion models by regressing onto a conditional score, although generation ultimately requires the marginal score. The two objectives share the same population minimizer, but the conditional target remains random at fixed noisy state and introduces an irreducible excess in the training loss. We isolate this excess and show that, for a general corruption kernel under mild regularity assumptions, it is exactly the trace of the Fisher--Rao metric of the conditional endpoint family, integrated along the diffusion trajectory. This gives an exact conditional-variance decomposition of the denoising objective and identifies the information geometry observed in diffusion latent spaces as an intrinsic component of the training loss. We derive the result from a Schr"odinger bridge variational principle, in which the ideal objective arises as excess path-space relative entropy. For corruption diffusions, the Fisher term is proportional to the rate at which the noisy state loses mutual information about the clean data, separating the loss floor into an information flow determined by the data and a weight determined by the corruption schedule and objective. In the Gaussian case, this yields a closed form for the floor, recovers reparametrization invariance of the continuous-time objective, and relates its high-SNR divergence to the information dimension of the data. Finally, we show that raw losses obtained with different noise ranges or weightings need not rank models consistently because they contain different additive floors, and contrast the second-order geometry seen by training with the third-order conditional statistics entering numerical sampling error.
Machine Learning (cs.LG); Statistical Mechanics (cond-mat.stat-mech)
arXiv:2608.23916 [cs.LG]
(or arXiv:2608.23916v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.23916 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 344 words
Related coverage
Supervised Learning Has a Geometric Blind Spot
arXiv · Aug 7, 2026
Connecting Score Matching, Maximum Likelihood, and Expectation-Maximization in Mixed Linear Regression
arXiv · Sep 9, 2026
Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacement
arXiv · Sep 30, 2026
Reducing the Adaptation Gap Through Reachable Fisher Geometry
arXiv · Sep 30, 2026