aiqbench analysis
This research investigates the stability of human reasoning patterns over time, proposing a geometry-based method to quantify internal variability and its interaction with task correctness. By demonstrating that these cognitive patterns remain stable within individuals, the study offers a formal framework for modeling learning effects in cognitive science. The findings provide a theoretical foundation for aligning AI systems more closely with the structural consistency of human reasoning processes.
Response Variability and Stability in Human Reasoning
arXiv:2610.03008v1 Announce Type: cross Abstract: Understanding how humans reason -- and how reasoning responses vary across tasks and individuals -- remains a core challenge for modeling and explanation in cognitive science. We investigate the stability of response patterns within reasoners and whether variation in these patterns can be used to predict learning effects. We introduce a formal, geometry-based method to quantify distances between individual reasoning patterns and their internal variability, grounded in heuristic theories. The proposed framework is tested against experimental data via generalized linear mixed-effects models and clustering, where we find that our proposed variation measure interacts with correctness to predict performance gains. Moreover, we find that reasoning patterns are stable over time within the same reasoner. The method is general enough to be applied to other reasoning domains.
Abstract:Understanding how humans reason -- and how reasoning responses vary across tasks and individuals -- remains a core challenge for modeling and explanation in cognitive science. We investigate the stability of response patterns within reasoners and whether variation in these patterns can be used to predict learning effects. We introduce a formal, geometry-based method to quantify distances between individual reasoning patterns and their internal variability, grounded in heuristic theories. The proposed framework is tested against experimental data via generalized linear mixed-effects models and clustering, where we find that our proposed variation measure interacts with correctness to predict performance gains. Moreover, we find that reasoning patterns are stable over time within the same reasoner. The method is general enough to be applied to other reasoning domains.
Accepted as full paper with talk at the 48th Annual Conference of the Cognitive Science Society (CogSci 2026). This version contains minor revisions
Logic in Computer Science (cs.LO); Artificial Intelligence (cs.AI); Neurons and Cognition (q-bio.NC)
arXiv:2610.03008 [cs.LO]
(or arXiv:2610.03008v1 [cs.LO] for this version)
https://doi.org/10.48550/arXiv.2610.03008 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 249 words
Related coverage

[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · Sep 17, 2026

Why do models task game?
AI Alignment Forum · Aug 6, 2026

Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT
METR · Jul 21, 2026
Simulated Self-Assessment in Large Language Models: A Psychometric Approach to AI Self-Efficacy
arXiv · Aug 27, 2026