Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures

arXiv:2606.19894v2 Announce Type: replace Abstract: Score-based diffusion models have achieved remarkable empirical success, motivating extensive theoretical work to establish their foundations. However, existing complexity bounds for score approximation, a vital step in diffusion modeling, rely on rigid constraints such as Lipschitz continuous scores or lower bounded densities. This severely limits their applicability to real-world perceptual data, where singularities, sharp boundaries, and disjoint clusters routinely violate such restrictive assumptions. We bridge this gap between theory and practice, presenting the first universal score approximation theorem applicable to any compactly supported distribution in $\mathbb{R}^n$. Using a novel discretization technique that directly models the underlying distribution, we prove that the neural network complexity is governed by the support's upper Minkowski dimension $d$ rather than the ambient dimension $n$. Furthermore, by leveraging the inherent smoothing of Gaussian kernels, we show that even for irregular, fractal distributions, an $\epsilon$ approximation error can be achieved with $\mathcal{O}(\epsilon^{-{d/(M+1)}})$ local Taylor modules each at size of $\mathcal{O}(n^M)$, an $\epsilon$-scaling rate previously achieved only for distributions with $(M+1)$-H\"older smoothness. Thus, we reveal a possible mechanism by which score-based diffusion models represent non-smooth data distributions.

Abstract:Score-based diffusion models have achieved remarkable empirical success, motivating extensive theoretical work to establish their foundations. However, existing complexity bounds for score approximation, a vital step in diffusion modeling, rely on rigid constraints such as Lipschitz continuous scores or lower bounded densities. This severely limits their applicability to real-world perceptual data, where singularities, sharp boundaries, and disjoint clusters routinely violate such restrictive assumptions. We bridge this gap between theory and practice, presenting the first universal score approximation theorem applicable to any compactly supported distribution in $\mathbb{R}^n$. Using a novel discretization technique that directly models the underlying distribution, we prove that the neural network complexity is governed by the support's upper Minkowski dimension $d$ rather than the ambient dimension $n$. Furthermore, by leveraging the inherent smoothing of Gaussian kernels, we show that even for irregular, fractal distributions, an $\epsilon$ approximation error can be achieved with $\mathcal{O}(\epsilon^{-{d/(M+1)}})$ local Taylor modules each at size of $\mathcal{O}(n^M)$, an $\epsilon$-scaling rate previously achieved only for distributions with $(M+1)$-Hölder smoothness. Thus, we reveal a possible mechanism by which score-based diffusion models represent non-smooth data distributions.

Machine Learning (cs.LG)

arXiv:2606.19894 [cs.LG]

(or arXiv:2606.19894v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2606.19894 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

analiza aiqbench

Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures

Subiecte conexe