Diffusion Operator Geometry of Feedforward Representations

arXiv:2605.01107v2 Announce Type: replace Abstract: Feedforward neural networks transform data through learned representations whose geometry shapes how classes separate and relate across successive layers. We study that geometry through diffusion operators. Each feature-cloud snapshot is assigned a Gaussian-kernel Markov operator, giving a smooth description of one-step transport between classes from which spectral, boundary, and local geometric information can be read. We define the empirical class chain, state the condition under which it is an exact Markov quotient, and derive both the corresponding population transition and a simpler overlap chain based on expected class affinities. For balanced shared-covariance Gaussian class-conditional snapshots these affinities have closed forms controlled by a regularized Mahalanobis separation, which yields explicit expressions for leakage and coarse spectral behaviour. We further show that operator observables vary smoothly under feature perturbations, whereas hard neighborhood graphs are controlled by neighbor-order margins. Experiments on CIFAR-10 and CIFAR-100 ResNet-18 representations find that class transport becomes increasingly persistent with depth while retaining structured relations between classes, and that the diffusion class chain is more stable than its $k$-nearest-neighbor counterpart under matched perturbations.

Abstract:Feedforward neural networks transform data through learned representations whose geometry shapes how classes separate and relate across successive layers. We study that geometry through diffusion operators. Each feature-cloud snapshot is assigned a Gaussian-kernel Markov operator, giving a smooth description of one-step transport between classes from which spectral, boundary, and local geometric information can be read. We define the empirical class chain, state the condition under which it is an exact Markov quotient, and derive both the corresponding population transition and a simpler overlap chain based on expected class affinities. For balanced shared-covariance Gaussian class-conditional snapshots these affinities have closed forms controlled by a regularized Mahalanobis separation, which yields explicit expressions for leakage and coarse spectral behaviour. We further show that operator observables vary smoothly under feature perturbations, whereas hard neighborhood graphs are controlled by neighbor-order margins. Experiments on CIFAR-10 and CIFAR-100 ResNet-18 representations find that class transport becomes increasingly persistent with depth while retaining structured relations between classes, and that the diffusion class chain is more stable than its $k$-nearest-neighbor counterpart under matched perturbations.

Machine Learning (cs.LG); Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (stat.ML)

arXiv:2605.01107 [cs.LG]

(or arXiv:2605.01107v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2605.01107 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

Diffusion Operator Geometry of Feedforward Representations

Related coverage