Local Stability and Gaussian Smoothing of Quantized Neural Networks

Abstract:We study Gaussian averaging as a smooth surrogate for quantized neural models. Under bounded local oscillation, we derive a local dimension-dependent bound on |f-g|, linking Gaussian smoothing to the stability analysis of discontinuous networks. We compute closed-form Gaussian averages of the rectified linear unit (ReLU) and sign activation functions, and illustrate the mechanism on a high-dimensional binary perceptron, where layer-preactivation aggregation under an explicit quantization-noise surrogate yields the Gaussian envelope used in inference-side smoothing and training-side smooth surrogate gradients.

Machine Learning (cs.LG); Systems and Control (eess.SY); Optimization and Control (math.OC)

I.2.6; I.5.1; I.2.8; G.1.2

arXiv:2607.20153 [cs.LG]

(or arXiv:2607.20153v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2607.20153 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.