aiqbench analysis
This paper presents a mathematical framework analyzing how deep linear networks develop compositional reasoning, linking the statistical structure of training data to emergent cognitive capabilities. By deriving conditions under which new inferences unlock from existing evidence, the work offers theoretical insights into the mechanics of generalization and abstraction in machine learning systems. It addresses fundamental questions about when and why models can fail or succeed at combining known premises, providing a rigorous explanation for phenomena observed in both synthetic environments and human cognition.
A Spectral Theory of Compositional Learning
arXiv:2609.33708v1 Announce Type: new Abstract: How does compositional reasoning emerge during learning? We address this question by mathematically analyzing the learning dynamics of deep linear networks. We train these networks in structured synthetic environments and derive a theory linking the structure of experience to compositional learning. Our theory predicts when compositional inferences emerge, whether they are identifiable from the available evidence, and how new linking evidence can rapidly unlock previously unavailable inferences. These results provide a qualitative explanation for several phenomena observed in human cognition. They account for why a composition can fail despite knowing its premises, why similar compositions can emerge at different times, and how a single linking fact can suddenly enable many new inferences. Taken together, these findings establish a mathematical link between the statistical structure of experience and the development of compositional reasoning.
Abstract:How does compositional reasoning emerge during learning? We address this question by mathematically analyzing the learning dynamics of deep linear networks. We train these networks in structured synthetic environments and derive a theory linking the structure of experience to compositional learning. Our theory predicts when compositional inferences emerge, whether they are identifiable from the available evidence, and how new linking evidence can rapidly unlock previously unavailable inferences. These results provide a qualitative explanation for several phenomena observed in human cognition. They account for why a composition can fail despite knowing its premises, why similar compositions can emerge at different times, and how a single linking fact can suddenly enable many new inferences. Taken together, these findings establish a mathematical link between the statistical structure of experience and the development of compositional reasoning.
22 pages, 9 figures, and 2 tables. Under review at ICLR 2027
Machine Learning (cs.LG)
arXiv:2609.33708 [cs.LG]
(or arXiv:2609.33708v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.33708 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 238 words
Related coverage
Deep Embedded Multiplicative DMD for Algebra-Preserving Koopman Learning
arXiv · Jun 5, 2026
Exploring Interpretability in Deep Learning-Based Forest Classification - Separating Similar Forest Classes using Sentinel-2 Time Series
arXiv · Oct 15, 2026
Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
arXiv · Sep 7, 2026
Reflections on unlearning and inoculation
LessWrong · Sep 20, 2026