analiza aiqbench
This research provides a rigorous theoretical framework for understanding chain-of-thought reasoning in LLMs by modeling the discovery process as mean-field dynamics on a clue graph, avoiding physical analogies to preserve architectural fidelity. By deriving an ordinary differential equation that fits statistical regularities across multiple reasoning chains, the study offers actionable insights into optimizing model behavior and interpreting internal token generation mechanisms without relying on simplified architectures.
Mean-Field Dynamics of Chain-of-Thought Reasoning in Large Language Models
arXiv:2608.05152v1 Announce Type: new Abstract: Large language models (LLMs) with chain-of-thought reasoning have been widely applied in recent years, and theoretical explanations of their behavior may help deepen our understanding and guide model optimization. In this study, we introduce a framework that seeks statistical regularities and theoretical interpretations in LLM reasoning without simplifying the model architecture or making analogies to existing physical systems. We formulate LLM reasoning as a guided discovery process on a clue graph, and derive a one-dimensional ordinary differential equation for the fraction of discovered clues using the mean-field approximation. Experimentally, clue tokens are identified using the normalized surprisal of a student LLM on the outputs of a teacher LLM, and statistical regularities are obtained by averaging over many reasoning chains of thought. Our experiments show that the resulting statistical regularities are reproducible within the same dataset and can be fitted by the solving the proposed theoretical equation.
Abstract:Large language models (LLMs) with chain-of-thought reasoning have been widely applied in recent years, and theoretical explanations of their behavior may help deepen our understanding and guide model optimization. In this study, we introduce a framework that seeks statistical regularities and theoretical interpretations in LLM reasoning without simplifying the model architecture or making analogies to existing physical systems. We formulate LLM reasoning as a guided discovery process on a clue graph, and derive a one-dimensional ordinary differential equation for the fraction of discovered clues using the mean-field approximation. Experimentally, clue tokens are identified using the normalized surprisal of a student LLM on the outputs of a teacher LLM, and statistical regularities are obtained by averaging over many reasoning chains of thought. Our experiments show that the resulting statistical regularities are reproducible within the same dataset and can be fitted by the solving the proposed theoretical equation.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2608.05152 [cs.CL]
(or arXiv:2608.05152v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.05152 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 242 cuvinte
Subiecte conexe
Links #5: 2026/07
LessWrong · 31 iul. 2026
MedPAIR: Measuring Whether Physicians and AI Agree on What Matters in Medical QA
Snorkel AI Blog · 2 oct. 2026
The world's best gradual disempowerment model organism: Frontier AI labs
LessWrong · 29 sept. 2026
How might continual learning affect safety and alignment?
LessWrong · 13 iun. 2026