aiqbench analysis
This research clarifies the relationship between architectural sparsity in Kolmogorov-Arnold Networks (KANs) and statistical Fisher nullity, demonstrating that zero coefficients do not always equate to redundant parameters in these models. The study establishes that effective path disconnections are required to identify truly invisible directions, distinguishing them from mere coefficient magnitude thresholds used in standard multilayer perceptrons. These findings provide a rigorous diagnostic framework for pruning KANs without compromising model rank or interpretability guarantees.
Fisher Simplicity in Kolmogorov-Arnold Networks and Multilayer Perceptrons
arXiv:2609.32503v1 Announce Type: new Abstract: Kolmogorov-Arnold Networks (KANs) are motivated in part by interpretability: their learned edge functions can be inspected, pruned, and reduced to symbolic structure. In a fixed-basis KAN, this makes a small or zero basis coefficient look like a certificate of simplicity, much as a dead rectified linear unit (ReLU) marks unused computation in a multilayer perceptron (MLP). Fisher nullity gives a precise statistical notion: a parameter direction is Fisher-simple exactly when perturbing it is invisible under the task distribution. We study when these architectural and Fisher notions agree. For a dead ReLU unit, they agree: the closed activation region makes the associated score directions vanish. For a fixed-basis KAN, they do not. In the single-layer Gaussian case, the coefficient Fisher matrix is a basis Gram matrix under the input distribution and is independent of the fitted coefficients. In a multilayer KAN, Fisher simplicity is graph-path based: the data must reach a basis atom and its perturbation must propagate through the downstream network. We encode these two conditions in an effective edge measure and, under local dictionary independence and effective-measure nondegeneracy, show that zero effective exposure exactly identifies Fisher-null directions within an edge. Controlled diagnostics confirm that zero coefficients can preserve rank while effective path disconnections remove the predicted directions. Coefficient magnitude alone is therefore not a Fisher-based pruning criterion for KANs.
Abstract:Kolmogorov-Arnold Networks (KANs) are motivated in part by interpretability: their learned edge functions can be inspected, pruned, and reduced to symbolic structure. In a fixed-basis KAN, this makes a small or zero basis coefficient look like a certificate of simplicity, much as a dead rectified linear unit (ReLU) marks unused computation in a multilayer perceptron (MLP). Fisher nullity gives a precise statistical notion: a parameter direction is Fisher-simple exactly when perturbing it is invisible under the task distribution. We study when these architectural and Fisher notions agree. For a dead ReLU unit, they agree: the closed activation region makes the associated score directions vanish. For a fixed-basis KAN, they do not. In the single-layer Gaussian case, the coefficient Fisher matrix is a basis Gram matrix under the input distribution and is independent of the fitted coefficients. In a multilayer KAN, Fisher simplicity is graph-path based: the data must reach a basis atom and its perturbation must propagate through the downstream network. We encode these two conditions in an effective edge measure and, under local dictionary independence and effective-measure nondegeneracy, show that zero effective exposure exactly identifies Fisher-null directions within an edge. Controlled diagnostics confirm that zero coefficients can preserve rank while effective path disconnections remove the predicted directions. Coefficient magnitude alone is therefore not a Fisher-based pruning criterion for KANs.
8 pages, 2 figures. Presented at Allerton 2026
Machine Learning (cs.LG)
arXiv:2609.32503 [cs.LG]
(or arXiv:2609.32503v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.32503 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 322 words
Related coverage
Evaluation of Sampling Strategies and Physics-Informed Kolmogorov--Arnold Networks in Unbounded Domains
arXiv · Oct 5, 2026
SechKAN: Kolmogorov-Arnold Networks with Hyperbolic Secant Functions
arXiv · Jul 23, 2026
PG-KINN: A Physics-Informed Petrov-Galerkin Kolmogorov-Arnold Network for Solving Forward and Inverse PDEs
arXiv · Jul 23, 2026
KANLib -- An Modular, Extensible and Fast Kolmogorov-Arnold Network Implementation
arXiv · Jun 17, 2026