analiza aiqbench
This research introduces "causal pieces," a theoretical framework for analyzing spiking neural networks that partitions input spaces into regions governed by consistent subnetworks. The study establishes that the count of these pieces serves as a metric for approximation capability and correlates strongly with training success across multiple benchmarks. By bridging discontinuous spike-time dynamics with Lipschitz continuity, this work offers a principled approach to improving the expressivity and trainability of SNNs.
Causal pieces: analysing and improving spiking neural networks piece by piece
arXiv:2504.14015v2 Announce Type: replace-cross Abstract: We introduce "causal pieces", a novel concept for analysing spiking neural networks (SNNs), inspired by "linear pieces" used to study expressivity and trainability in artificial neural networks (ANNs). Causal pieces partition the input and parameter space of a feedforward SNN with single-spike coding into distinct regions where the same subnetwork causes the output spikes. For networks of current-based leaky integrate-and-fire (LIF) neurons with large membrane time constants, we show that within each causal piece, output spike times are locally Lipschitz continuous with respect to inputs and network parameters. We further prove a lower bound on the approximation error that depends on the number of causal pieces. Thus, the number of causal pieces is a measure of the approximation capabilities of SNNs, which is valid despite spike-time discontinuities and applies to networks with both excitatory and inhibitory synapses. Empirically, we find that parameter initialisations yielding more causal pieces on the training set strongly correlate with SNN training success across multiple benchmarks, including Yin-Yang, Fashion-MNIST, and EuroSAT. Moreover, simulations with standard single-spike LIF neurons indicate that our findings extend beyond the theoretically analysed regime. These results establish causal pieces as a powerful and principled tool for analysing and improving the computational capabilities of SNNs.
Abstract:We introduce "causal pieces", a novel concept for analysing spiking neural networks (SNNs), inspired by "linear pieces" used to study expressivity and trainability in artificial neural networks (ANNs). Causal pieces partition the input and parameter space of a feedforward SNN with single-spike coding into distinct regions where the same subnetwork causes the output spikes. For networks of current-based leaky integrate-and-fire (LIF) neurons with large membrane time constants, we show that within each causal piece, output spike times are locally Lipschitz continuous with respect to inputs and network parameters. We further prove a lower bound on the approximation error that depends on the number of causal pieces. Thus, the number of causal pieces is a measure of the approximation capabilities of SNNs, which is valid despite spike-time discontinuities and applies to networks with both excitatory and inhibitory synapses. Empirically, we find that parameter initialisations yielding more causal pieces on the training set strongly correlate with SNN training success across multiple benchmarks, including Yin-Yang, Fashion-MNIST, and EuroSAT. Moreover, simulations with standard single-spike LIF neurons indicate that our findings extend beyond the theoretically analysed regime. These results establish causal pieces as a powerful and principled tool for analysing and improving the computational capabilities of SNNs.
Accepted for publication at NeurIPS 2026. Code repository: this https URL
Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neurons and Cognition (q-bio.NC); Machine Learning (stat.ML)
arXiv:2504.14015 [cs.NE]
(or arXiv:2504.14015v2 [cs.NE] for this version)
Text integral sub licența deschisă a publicației · 239 cuvinte
Subiecte conexe
Polyhedral Geometry of Time-to-First-Spike Neural Networks
arXiv · 11 sept. 2026
Upper bounds on the number of causal pieces in spiking neural networks
arXiv · 19 oct. 2026
Increasing Skill Level Recruits Deeper Attention Layers in a Frozen Chess Transformer
LessWrong · 24 sept. 2026
How Does Changing the Elo of a Chess Transformer Affect its Computations?
LessWrong · 29 sept. 2026