aiqbench analysis
This paper links an LLM's ability to make novel scientific discoveries to "monofacts," facts seen only once in training data, and tests the link by manipulating monofact rates in materials science and regulatory genomics. The effect holds across two open-weight model families and different sampling and reasoning settings, and the authors report it is not explained by diversity alone. For readers, it offers a measurable, data-side hypothesis for when models can generate new scientific findings, though the evidence comes from two domains and two model families.
Tracing the Scientific Discovery Capacity of LLMs to Their Training Data
arXiv:2611.03617v1 Announce Type: cross Abstract: Large language models (LLMs) show growing promise for scientific discovery, yet basic questions remain about what enables reliable discovery. How can we trace the scientific discovery capacity of LLMs to their training data? We study this question as one of missing probability mass and develop a theoretical account in which monofacts-the facts observed once during training-set the discovery capacity. In materials science and regulatory genomics, we directly manipulate monofact rates in training sets and find that more monofacts consistently yield more discoveries across two open-weight model families, sampling strategies, and reasoning budgets. Further experiments show the relationship cannot be explained by training-set diversity alone and can transfer across scientific domains when paired with in-domain data. Together, our results identify monofact rate in training data as a key factor governing discoveries in language models.
Abstract:Large language models (LLMs) show growing promise for scientific discovery, yet basic questions remain about what enables reliable discovery. How can we trace the scientific discovery capacity of LLMs to their training data? We study this question as one of missing probability mass and develop a theoretical account in which monofacts-the facts observed once during training-set the discovery capacity. In materials science and regulatory genomics, we directly manipulate monofact rates in training sets and find that more monofacts consistently yield more discoveries across two open-weight model families, sampling strategies, and reasoning budgets. Further experiments show the relationship cannot be explained by training-set diversity alone and can transfer across scientific domains when paired with in-domain data. Together, our results identify monofact rate in training data as a key factor governing discoveries in language models.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2611.03617 [cs.CL]
(or arXiv:2611.03617v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2611.03617 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 231 words
Related coverage
Beyond Equation Search: Conjecture-Guided Discovery with SCOPE and ClosureBench
arXiv · Oct 15, 2026
Principia: Test-Time Hypotheses to Principle Construction for Scientific Discovery
arXiv · Oct 27, 2026
Can LLMs Discover Scientific Laws in Real and Parallel Worlds?
arXiv · Sep 2, 2026
The Scientific Contribution Graph: Automated Literature-based Technological Roadmapping at Scale
arXiv · Sep 2, 2026