aiqbench analysis
This paper advances diffusion model training by aligning hidden states with self-supervised spectral representations, revealing a geometric equivalence to score distillation. By integrating a spectral regularizer into the training objective, the authors demonstrate consistent improvements in generation quality across images and 3D point clouds without relying on external data. The work provides critical theoretical insight into how representation alignment mechanisms enhance convergence and sampling efficiency in generative AI.
Revisiting Spectral Representations in Generative Diffusion Models
arXiv:2609.08253v1 Announce Type: new Abstract: Diffusion models have shown remarkable performance on diverse generation tasks. Recent work finds that imposing representation alignment on the hidden states of diffusion networks can both facilitate training convergence and enhance sampling quality, yet the mechanism driving this synergy remains insufficiently understood. In this paper, we investigate the connection between self-supervised spectral representation learning and diffusion generative models through a shared perspective on perturbation kernels. On the diffusion side, samples (e.g., images, videos) are produced by reversing a stochastic noise-injection process specified by Gaussian kernels; on the spectral representation side, spectral embeddings emerge from contrasting positive and negative relations induced by random perturbation kernels. Motivated by this, we propose a self-supervised spectral representation alignment method to facilitate diffusion model training. In addition, we clarify how joint spectral learning can benefit diffusion training from a geometric perspective. Furthermore, we find that the optimization of the spectral alignment objective is in an equivalent form of diffusion score distillation in the representation space. Building on these findings, we integrate a spectral regularizer into diffusion training objectives to improve the performance of diffusion models on multiple datasets. Experiments across images and 3D point clouds show consistent gains in generation quality. Code is released at https://github.com/yuehaowang/spectral-reg-diffusion.
Abstract:Diffusion models have shown remarkable performance on diverse generation tasks. Recent work finds that imposing representation alignment on the hidden states of diffusion networks can both facilitate training convergence and enhance sampling quality, yet the mechanism driving this synergy remains insufficiently understood. In this paper, we investigate the connection between self-supervised spectral representation learning and diffusion generative models through a shared perspective on perturbation kernels. On the diffusion side, samples (e.g., images, videos) are produced by reversing a stochastic noise-injection process specified by Gaussian kernels; on the spectral representation side, spectral embeddings emerge from contrasting positive and negative relations induced by random perturbation kernels. Motivated by this, we propose a self-supervised spectral representation alignment method to facilitate diffusion model training. In addition, we clarify how joint spectral learning can benefit diffusion training from a geometric perspective. Furthermore, we find that the optimization of the spectral alignment objective is in an equivalent form of diffusion score distillation in the representation space. Building on these findings, we integrate a spectral regularizer into diffusion training objectives to improve the performance of diffusion models on multiple datasets. Experiments across images and 3D point clouds show consistent gains in generation quality. Code is released at this https URL.
Machine Learning (cs.LG)
arXiv:2609.08253 [cs.LG]
(or arXiv:2609.08253v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.08253 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 297 words
Related coverage
Spectral Feedback for Test-Time Alignment of Protein Diffusion Models
arXiv · Sep 28, 2026
Physics-informed diffusion models in spectral space
arXiv · Jun 3, 2026
Geometry-Guided Generative Representation for Functional Brain Graphs
arXiv · Jul 23, 2026
Spectrally Regularized Latent Flow Matching for Turbulence Generation
arXiv · Jun 12, 2026