aiqbench analysis
URCHIN represents a significant step in biological plausibility by demonstrating that language modeling can be achieved with minimal parameters and no attention mechanisms, relying instead on spiking neurons and lateral connectomes. This approach offers a viable path toward energy-efficient inference on neuromorphic hardware while maintaining performance comparable to larger models on constrained datasets.
URCHIN: A Horizontal Spiking Language Model for Data-Constrained Pretraining
arXiv:2609.13899v1 Announce Type: cross Abstract: The BabyLM challenge measures how much language a model can learn from developmentally-plausible, child-scale data rather than internet-scale corpora, yet prior language models forgo the biological constraints of the neural circuitry that acquires human language: spiking neurons separated into excitatory and inhibitory populations wired by a recurrent lateral connectome. This paper presents URCHIN (Unified Recurrent Connectome with Horizontal Integrate-and-fire Neurons), which applies the Parallelized Hierarchical Connectome Spiking State-space Model (PHCSSM) to language modeling: leaky integrate-and-fire neurons coupled by a Dale's-law lateral connectome resolve each token through a multi-transmission loop that recirculates activity to a fixed point. The instantiation is deliberately minimal: a single horizontal layer of 128 neurons, no attention, and 4.23M parameters. Two implementations share one set of weights and produce identical benchmark scores, so URCHIN is trained once and deployed either way with no conversion step: a parallel state-space model (SSM) scan that is GPU-efficient for training, or an event-driven recurrent spiking neural network (RSNN) with constant-cost inference for CPU or neuromorphic edge deployment. Across all three BabyLM tracks (Strict-100M, Strict-Small, and Multilingual), URCHIN offers a biologically plausible, efficient, and directly deployable reference point.
Abstract:The BabyLM challenge measures how much language a model can learn from developmentally-plausible, child-scale data rather than internet-scale corpora, yet prior language models forgo the biological constraints of the neural circuitry that acquires human language: spiking neurons separated into excitatory and inhibitory populations wired by a recurrent lateral connectome. This paper presents URCHIN (Unified Recurrent Connectome with Horizontal Integrate-and-fire Neurons), which applies the Parallelized Hierarchical Connectome Spiking State-space Model (PHCSSM) to language modeling: leaky integrate-and-fire neurons coupled by a Dale's-law lateral connectome resolve each token through a multi-transmission loop that recirculates activity to a fixed point. The instantiation is deliberately minimal: a single horizontal layer of 128 neurons, no attention, and 4.23M parameters. Two implementations share one set of weights and produce identical benchmark scores, so URCHIN is trained once and deployed either way with no conversion step: a parallel state-space model (SSM) scan that is GPU-efficient for training, or an event-driven recurrent spiking neural network (RSNN) with constant-cost inference for CPU or neuromorphic edge deployment. Across all three BabyLM tracks (Strict-100M, Strict-Small, and Multilingual), URCHIN offers a biologically plausible, efficient, and directly deployable reference point.
12 pages, 2 figures, 6 tables. Accepted to the BabyLM Challenge 2026 (BabyLM Workshop, EMNLP 2026)
Neurons and Cognition (q-bio.NC); Computation and Language (cs.CL)
I.2.7; I.2.6; I.5.1
arXiv:2609.13899 [q-bio.NC]
(or arXiv:2609.13899v1 [q-bio.NC] for this version)
https://doi.org/10.48550/arXiv.2609.13899 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 304 words
Related coverage
SpikeOPD: Stable On-Policy Distillation for Autoregressive Spiking Language Models
arXiv · Sep 1, 2026
Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws
arXiv · Jun 10, 2026
Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws
arXiv · Jun 8, 2026
Data Augmentations for Data-Constrained Language Model Pretraining
arXiv · Jun 17, 2026