aiqbench analysis
This paper establishes a rigorous, activation-independent Fourier-block principle that determines the optimal $L_2$ approximation rates of shallow neural networks for functions in mixed Sobolev spaces. By deriving matching algebraic lower and upper bounds for specific activations like ReLU$^k$, ELU, and cosine, it clarifies how network width and neuron count interact with function smoothness to dictate convergence speed. These theoretical limits are essential for understanding the fundamental representational capacity of shallow architectures in high-dimensional approximation tasks.
Shallow neural network approximation in mixed Sobolev spaces
arXiv:2609.05263v1 Announce Type: cross Abstract: We investigate the best $L_2$ approximation of mixed Sobolev spaces by shallow neural networks with $n$ neurons and general activation functions. We first establish an activation-independent Fourier-block principle: if an activation has univariate approximation order $\rho$ in the sense of the Fourier-block property, then the global approximation rate has algebraic order $\min\{\alpha,\rho\}$ for target functions of mixed smoothness $\alpha$, up to explicit logarithmic factors. To verify this property for concrete activations, we introduce a structured univariate approximation condition that implies the Fourier-block property with explicit parameters. For $\mathrm{ReLU}^k$, a matching algebraic lower bound identifies $\min\{\alpha,k+1\}$ as the optimal algebraic approximation exponent in any dimension, up to logarithmic factors in the upper bound. The framework also yields the exponent $\min\{\alpha,k+1\}$ for cardinal B-splines and soft-$\mathrm{ReLU}^k$, and the full mixed-smoothness exponent $\alpha$ for ELU and cosine activations, again up to logarithmic~factors.
Abstract:We investigate the best $L_2$ approximation of mixed Sobolev spaces by shallow neural networks with $n$ neurons and general activation functions. We first establish an activation-independent Fourier-block principle: if an activation has univariate approximation order $\rho$ in the sense of the Fourier-block property, then the global approximation rate has algebraic order $\min\{\alpha,\rho\}$ for target functions of mixed smoothness $\alpha$, up to explicit logarithmic factors. To verify this property for concrete activations, we introduce a structured univariate approximation condition that implies the Fourier-block property with explicit parameters. For $\mathrm{ReLU}^k$, a matching algebraic lower bound identifies $\min\{\alpha,k+1\}$ as the optimal algebraic approximation exponent in any dimension, up to logarithmic factors in the upper bound. The framework also yields the exponent $\min\{\alpha,k+1\}$ for cardinal B-splines and soft-$\mathrm{ReLU}^k$, and the full mixed-smoothness exponent $\alpha$ for ELU and cosine activations, again up to logarithmic~factors.
Numerical Analysis (math.NA); Machine Learning (cs.LG)
arXiv:2609.05263 [math.NA]
(or arXiv:2609.05263v1 [math.NA] for this version)
https://doi.org/10.48550/arXiv.2609.05263 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 236 words
Related coverage
Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networks
arXiv · Jul 27, 2026
Neural Network Approximation of Solutions to Fractional Parabolic Partial Differential Equations
arXiv · Jul 31, 2026
Manifold limit for the training of shallow graph convolutional neural networks
arXiv · Sep 30, 2026
Boundary-Adapted PINNs for Elliptic Dirichlet Problems: $H^2(\Omega)$ A Priori Error Bounds with Application to Mean Escape Time Computation
arXiv · Jul 22, 2026