aiqbench analysis
This paper proposes a single-hidden-layer network whose units contain learnable multivariate polynomials, aiming to match deep networks and Kolmogorov-Arnold networks with fewer parameters and lower training cost. The universal approximation proofs and convergence rates are the main theoretical contributions, while the empirical claims rest on comparisons across many benchmark functions and "comparable" parameter budgets, so practical gains will depend on how those baselines were tuned. Readers should watch whether the approach holds up on high-dimensional, real-world tasks beyond the synthetic function-approximation setting.
SUPN: Shallow Universal Polynomial Networks for Efficient Function Approximation
arXiv:2511.21414v2 Announce Type: replace Abstract: Deep neural networks (DNNs) and Kolmogorov-Arnold networks (KANs) are flexible function approximators, but their accuracy can require substantial network depth, width, and trainable parameter counts. Beyond making the resulting networks less transparent, large parameter spaces can increase training cost and produce solutions that are sensitive to optimization and parameter initialization, leading to variability in generalization error. For these reasons, we propose Shallow Universal Polynomial Networks (SUPNs), which retain a single hidden layer while increasing the expressivity of each unit through a learnable multivariate polynomial inside each activation, which we refer to as a polynomial pre-activation. This construction trades architectural depth and width for greater expressivity within each hidden unit, reducing parameter counts and training cost while preserving a shallow computational graph. We establish two complementary universal approximation results: SUPNs are universal by increasing network width at fixed polynomial complexity, as in classical shallow-network approximation, and by increasing polynomial complexity at fixed finite width. We further prove that finite SUPNs can approach the best approximation attainable in a prescribed polynomial space to arbitrary relative tolerance, derive explicit quasi-optimal parameters, and establish corresponding convergence rates. We empirically evaluate SUPNs against DNNs, KANs, and polynomial projection on smooth, nonsmooth, and discontinuous function-approximation problems in one, two, and ten dimensions using more than 16,000 trained models. Across the target functions studied, SUPNs frequently achieve lower approximation error and variability than DNNs and KANs at comparable parameter counts while retaining lower training cost, supporting their use as efficient function approximators when the target structure is not known a priori.
Abstract:Deep neural networks (DNNs) and Kolmogorov-Arnold networks (KANs) are flexible function approximators, but their accuracy can require substantial network depth, width, and trainable parameter counts. Beyond making the resulting networks less transparent, large parameter spaces can increase training cost and produce solutions that are sensitive to optimization and parameter initialization, leading to variability in generalization error. For these reasons, we propose Shallow Universal Polynomial Networks (SUPNs), which retain a single hidden layer while increasing the expressivity of each unit through a learnable multivariate polynomial inside each activation, which we refer to as a polynomial pre-activation. This construction trades architectural depth and width for greater expressivity within each hidden unit, reducing parameter counts and training cost while preserving a shallow computational graph. We establish two complementary universal approximation results: SUPNs are universal by increasing network width at fixed polynomial complexity, as in classical shallow-network approximation, and by increasing polynomial complexity at fixed finite width. We further prove that finite SUPNs can approach the best approximation attainable in a prescribed polynomial space to arbitrary relative tolerance, derive explicit quasi-optimal parameters, and establish corresponding convergence rates. We empirically evaluate SUPNs against DNNs, KANs, and polynomial projection on smooth, nonsmooth, and discontinuous function-approximation problems in one, two, and ten dimensions using more than 16,000 trained models. Across the target functions studied, SUPNs frequently achieve lower approximation error and variability than DNNs and KANs at comparable parameter counts while retaining lower training cost, supporting their use as efficient function approximators when the target structure is not known a priori.
Machine Learning (cs.LG); Numerical Analysis (math.NA)
arXiv:2511.21414 [cs.LG]
(or arXiv:2511.21414v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2511.21414 Focus to learn more arXiv-issued DOI via DataCite
Trans. Mach. Learn. Res. (2026), ISSN 2835-8856
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 356 words
Related coverage
Lattice-Based Cryptography Explained
Quantum Zeitgeist · Aug 15, 2026
A Mike's-Eye View of ARC's Research
AI Alignment Forum · Jun 9, 2026
A brief history of quantum vs classical computational advantage
Quantum Journal · Sep 1, 2026
Researchers Decode up to 1,023 Qubits Using Rank Decomposition
Quantum Zeitgeist · Oct 3, 2026