aiqbench analysis
This study investigates the limits of neural networks in mathematical discovery by testing whether language models can independently derive the concept of zero without prior exposure. The findings indicate that while GPT-2-sized models cannot generalize this concept at test time, they achieve significant improvement after supervised training on specific examples, with linguistic pretraining reducing sample requirements by half. These results clarify how language capabilities scaffold arithmetic reasoning and define the data efficiency needed for AI to expand into new mathematical domains.
Nothing from Something: Can a Language Model Discover 0?
arXiv:2606.17289v1 Announce Type: new Abstract: AI systems based on artificial neural networks are being developed with aspirations of pushing the boundary of human mathematical knowledge. A key question for these systems is how much they can reach beyond their training data. Mathematical discovery requires a strong form of out of distribution generalization; the ability to hypothesize genuinely new - and potentially logically more powerful - mathematical structures. It has been hypothesized that language abilities support such generalizations in human cognition. In this work, we use simple arithmetic as a case study for examining how modern AI models could expand their mathematical horizons, evaluating whether these models can independently discover the concept of "zero". We show that We show that (1) language models of a GPT-2 size are unable to perform this generalization at test time regardless of language pretraining, but (2) models can improve substantially after training on tens or hundreds of examples of zero. Additionally, we find that language pretraining reduces the number of required examples by approximately $50\%$, showing that language abilities can scaffold mathematical discovery in neural models.
Title:Nothing from Something: Can a Language Model Discover 0?
Abstract:AI systems based on artificial neural networks are being developed with aspirations of pushing the boundary of human mathematical knowledge. A key question for these systems is how much they can reach beyond their training data. Mathematical discovery requires a strong form of out of distribution generalization; the ability to hypothesize genuinely new - and potentially logically more powerful - mathematical structures. It has been hypothesized that language abilities support such generalizations in human cognition. In this work, we use simple arithmetic as a case study for examining how modern AI models could expand their mathematical horizons, evaluating whether these models can independently discover the concept of "zero". We show that We show that (1) language models of a GPT-2 size are unable to perform this generalization at test time regardless of language pretraining, but (2) models can improve substantially after training on tens or hundreds of examples of zero. Additionally, we find that language pretraining reduces the number of required examples by approximately $50\%$, showing that language abilities can scaffold mathematical discovery in neural models.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2606.17289 [cs.AI]
(or arXiv:2606.17289v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.17289 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
https://doi.org/10.32470/vp0ddtg Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 293 words
Related coverage
Building a Foundation Stack for General-Purpose Robots
IEEE Spectrum · Jul 13, 2026
Presentation: From OTEL to SLMs: Distilling Frontier Model Behaviour from Production Telemetry
InfoQ · Jul 17, 2026
Nothing Changed but the Model: CellFill -- Bounded In-Cell Learning for Bit-Identical, Revocable Updates to Quantized LLMs
arXiv · Aug 24, 2026
Robust Critics: Defending LLMs Against Multi-Turn Attacks
arXiv · Jul 24, 2026