aiqbench analysis
This position paper argues that many LLM behaviours labelled either "stochastic parroting" or "advanced reasoning" are better explained as context-directed extrapolation from training-data priors. It proposes an evaluation checklist and design patterns for separating memorisation and basic extrapolation from genuine application-level generalisation. For researchers and deployers, the practical point is that claims about LLM reasoning should be tested against training-data overlap and out-of-domain performance, not taken at face value.
Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors
arXiv:2505.23323v3 Announce Type: replace Abstract: This position paper argues that many behaviours framed as either "stochastic parroting" or "advanced reasoning" are better explained as context-directed extrapolation from training-data priors, and that ML and NLP research communities would benefit from evaluations that distinguish remembering and basic extrapolation from genuine application-level generalisation. Our central contribution is an actionable evaluation checklist and design patterns that make these distinctions testable in practice. Our framework posits that LLMs extrapolate from training data priors while using context to guide the model to the appropriate priors; we call this "context-directed extrapolation." Specifically, this context direction is achieved through examples in base models, leading to in-context learning, while instruction tuning allows analogous task specification via prompts rather than explicit examples. We argue that adopting this more realistic framework and controls for improving evaluation would better calibrate claims about LLM "reasoning," refocus research on how LLM performance interacts with training data and generalisability to out-of-domain tasks, and reduce miscalibration in deployment and policy decisions.
Abstract:This position paper argues that many behaviours framed as either "stochastic parroting" or "advanced reasoning" are better explained as context-directed extrapolation from training-data priors, and that ML and NLP research communities would benefit from evaluations that distinguish remembering and basic extrapolation from genuine application-level generalisation. Our central contribution is an actionable evaluation checklist and design patterns that make these distinctions testable in practice. Our framework posits that LLMs extrapolate from training data priors while using context to guide the model to the appropriate priors; we call this "context-directed extrapolation." Specifically, this context direction is achieved through examples in base models, leading to in-context learning, while instruction tuning allows analogous task specification via prompts rather than explicit examples. We argue that adopting this more realistic framework and controls for improving evaluation would better calibrate claims about LLM "reasoning," refocus research on how LLM performance interacts with training data and generalisability to out-of-domain tasks, and reduce miscalibration in deployment and policy decisions.
Computation and Language (cs.CL)
arXiv:2505.23323 [cs.CL]
(or arXiv:2505.23323v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2505.23323 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 253 words