aiqbench analysis
This research investigates the geometric structure of convolutional neural networks using monomial activations to quantify model expressivity and analyze optimization critical points. By applying algebraic geometry tools, the study provides explicit formulas for neuromanifold properties that offer deeper theoretical insight into how these models learn from data.
On the Geometry and Optimization of Polynomial Convolutional Networks
arXiv:2410.00722v3 Announce Type: replace Abstract: We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere, up to rescaling the filters. By leveraging on tools from algebraic geometry, we explore the geometric properties of the image in function space of this map - typically referred to as neuromanifold. In particular, we compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model, and describe its singularities. Moreover, for a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.
Abstract:We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere, up to rescaling the filters. By leveraging on tools from algebraic geometry, we explore the geometric properties of the image in function space of this map - typically referred to as neuromanifold. In particular, we compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model, and describe its singularities. Moreover, for a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.
Accepted at AISTATS 2025. New version: corrected Section 4.2
Machine Learning (cs.LG); Algebraic Geometry (math.AG)
arXiv:2410.00722 [cs.LG]
(or arXiv:2410.00722v3 [cs.LG] for this version)
Full text under the publisher's open licence · 128 words
Related coverage
Convolutional Structure Cuts SPIM Complexity for Dense Problems
Quantum Zeitgeist · Aug 3, 2026
Few-Shot Prototype Head Adaptation for On-Device ECG Personalization on PSoC~6
arXiv · Oct 7, 2026
Muon Is Theoretically Wrong For Convolutions, But Empirically Effective
arXiv · Oct 7, 2026
Design of the IBM Granite 5.0 TurboCTC ASR Model
arXiv · Sep 18, 2026