aiqbench analysis
This study introduces the NeuroCognition benchmark to evaluate Large Language Models using neuropsychological tests that probe foundational cognitive abilities rather than just task completion. The findings reveal a distinct gap between current LLMs and human intelligence, particularly in abstract reasoning and spatial memory, while also demonstrating that complex reasoning strategies are not always superior to simple, human-like approaches. This research provides critical insights into the specific cognitive architectures where AI models still lack core adaptive cognition.
A Neuropsychologically Grounded Evaluation of LLM Cognitive Abilities
arXiv:2603.02540v2 Announce Type: replace Abstract: Large language models (LLMs) display a unified "general factor" of capability across 10 benchmarks (a finding confirmed by our factor analysis of 156 models), yet they still struggle with simple, trivial tasks for humans. This is because current benchmarks focus on task completion, failing to probe the foundational cognitive abilities that highlight these behaviors. We address this by introducing the NeuroCognition benchmark, grounded in three adapted neuropsychological tests targeting distinct foundational cognitive components: Raven's Progressive Matrices (abstract relational reasoning), Spatial Working Memory (goal-directed spatial updating), and the Wisconsin Card Sorting Test (cognitive flexibility). Our evaluation reveals that while models perform strongly on text, their performance degrades for images and with increased complexity. Comparison with a human baseline shows that LLMs and humans fail at different parts of the same tasks. Furthermore, we observe that complex reasoning is not universally beneficial, whereas simple, human-like strategies yield partial gains. We also find that NeuroCognition correlates positively with standard general-capability benchmarks, while still measuring distinct cognitive abilities beyond them. Overall, NeuroCognition emphasizes where current LLMs align with human-like intelligence and where they lack core adaptive cognition, showing the potential to serve as a verifiable, scalable source for improving LLMs.
Abstract:Large language models (LLMs) display a unified "general factor" of capability across 10 benchmarks (a finding confirmed by our factor analysis of 156 models), yet they still struggle with simple, trivial tasks for humans. This is because current benchmarks focus on task completion, failing to probe the foundational cognitive abilities that highlight these behaviors. We address this by introducing the NeuroCognition benchmark, grounded in three adapted neuropsychological tests targeting distinct foundational cognitive components: Raven's Progressive Matrices (abstract relational reasoning), Spatial Working Memory (goal-directed spatial updating), and the Wisconsin Card Sorting Test (cognitive flexibility). Our evaluation reveals that while models perform strongly on text, their performance degrades for images and with increased complexity. Comparison with a human baseline shows that LLMs and humans fail at different parts of the same tasks. Furthermore, we observe that complex reasoning is not universally beneficial, whereas simple, human-like strategies yield partial gains. We also find that NeuroCognition correlates positively with standard general-capability benchmarks, while still measuring distinct cognitive abilities beyond them. Overall, NeuroCognition emphasizes where current LLMs align with human-like intelligence and where they lack core adaptive cognition, showing the potential to serve as a verifiable, scalable source for improving LLMs.
Artificial Intelligence (cs.AI)
arXiv:2603.02540 [cs.AI]
(or arXiv:2603.02540v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2603.02540 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 288 words
Related coverage
[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · Sep 17, 2026
RAIL: Rethinking Auditory Intelligence in Large Audio-Language Models with a CHC-Grounded Benchmark
arXiv · Jun 11, 2026
Deconfusing Corrigibility
AI Alignment Forum · Oct 17, 2026
Almieyar-Oryx-BloomBench: A Bilingual Multimodal Benchmark for Cognitively Informed Evaluation of Vision-Language Models
arXiv · Jun 6, 2026