analiza aiqbench
This study establishes new theoretical bounds on the connectivity graphs of ReLU networks, revealing that their structural complexity is constrained by input dimension rather than network width or depth. These findings offer a rigorous geometric characterization of how linear regions partition input space, providing fresh insights into the fundamental behavior of piecewise-linear models. The work bridges a gap in understanding network topology beyond simple region counting, with implications for interpretability and architectural design.
Characterizing the Discrete Geometry of ReLU Networks
arXiv:2606.07728v1 Announce Type: new Abstract: It is well established that ReLU networks define continuous piecewise-linear functions, and that their linear regions are polyhedra in the input space. These regions form a complex that fully partitions the input space. The way these regions fit together is fundamental to the behavior of the network, as nonlinearities occur only at the boundaries where these regions connect. However, relatively little is known about the geometry of these complexes beyond bounds on the total number of regions, and calculating the complex exactly is intractable for most networks. In this work, we prove new theoretical results about these complexes that hold for all fully-connected ReLU networks, specifically about their connectivity graphs in which nodes correspond to regions and edges exist between each pair of regions connected by a face. We find that the average degree of this graph is upper bounded by twice the input dimension regardless of the width and depth of the network, and that the diameter of this graph has an upper bound that does not depend on input dimension, despite the number of regions increasing exponentially with input dimension. We corroborate our findings through experiments with networks trained on both synthetic and real-world data, which provide additional insight into the geometry of ReLU networks. Code to reproduce our results can be found at https://github.com/bl-ake/ICLR-2026.
Abstract:It is well established that ReLU networks define continuous piecewise-linear functions, and that their linear regions are polyhedra in the input space. These regions form a complex that fully partitions the input space. The way these regions fit together is fundamental to the behavior of the network, as nonlinearities occur only at the boundaries where these regions connect. However, relatively little is known about the geometry of these complexes beyond bounds on the total number of regions, and calculating the complex exactly is intractable for most networks. In this work, we prove new theoretical results about these complexes that hold for all fully-connected ReLU networks, specifically about their connectivity graphs in which nodes correspond to regions and edges exist between each pair of regions connected by a face. We find that the average degree of this graph is upper bounded by twice the input dimension regardless of the width and depth of the network, and that the diameter of this graph has an upper bound that does not depend on input dimension, despite the number of regions increasing exponentially with input dimension. We corroborate our findings through experiments with networks trained on both synthetic and real-world data, which provide additional insight into the geometry of ReLU networks. Code to reproduce our results can be found at this https URL.
Selected for an oral presentation at ICLR 2026. Tagged PDF, reviews, and discussions are available at this https URL
Machine Learning (cs.LG)
arXiv:2606.07728 [cs.LG]
(or arXiv:2606.07728v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.07728 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 332 cuvinte
Subiecte conexe
Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Space
arXiv · 21 iul. 2026
Theoretical Lower Bounds on the Robustness of Deep ReLU Networks
arXiv · 5 oct. 2026
Constraining the outputs of ReLU neural networks
arXiv · 16 iun. 2026
Exploding variance of means of exponentials: least-squares to the rescue – Machine Learning Research Blog
francisbach.com · 27 sept. 2026