aiqbench analysis
This study reveals a critical fragmentation in frontier vision-language models, where their Theory-of-Mind profiles diverge significantly between tasks rather than maintaining human-like consistency. The findings suggest that without explicit reasoning mechanisms like chain-of-thought, these systems exhibit distinct cognitive biases depending on the specific paradigm used for evaluation. For researchers building robust AI agents, this highlights the necessity of testing ToM capabilities across diverse benchmarks to avoid overestimating model coherence.
Cross-Task Dissociation in Frontier Vision-Language Model Theory of Mind
arXiv:2608.00261v1 Announce Type: cross Abstract: Do frontier vision-language models present a coherent Theory-of-Mind (ToM) profile across tasks, matching the same human reference group, or does that profile fragment from one paradigm to the next? We evaluate a shared panel of nine frontier VLMs on two psychology-derived benchmarks: the Keysar Director Task (visual perspective-taking under egocentric interference) and the Frith-Happ\'e animated triangles scored with the Castelli rubric (intention attribution from pure motion). On the Director Task, without chain-of-thought, the panel makes the egocentric error on 78\% of trials like children rather than adults; variation is substantial across models, and reasoning rescues several models. On the triangles, the panel under-attributes intention: its ToM profile sits more than three times closer to the high-functioning-autistic-adult (HF-ASD) mean than to the typical-development-adult (TD) mean, while Goal-Directed and Random stay near TD. No model is nearest TD on both tasks; the model that looks adult-like on the Director Task falls on the HF-ASD side on the triangles, and the most TD-like model on the triangles is child-like on the Director Task. We report group-level descriptions, not diagnostic labels for any model.
Abstract:Do frontier vision-language models present a coherent Theory-of-Mind (ToM) profile across tasks, matching the same human reference group, or does that profile fragment from one paradigm to the next? We evaluate a shared panel of nine frontier VLMs on two psychology-derived benchmarks: the Keysar Director Task (visual perspective-taking under egocentric interference) and the Frith-Happé animated triangles scored with the Castelli rubric (intention attribution from pure motion). On the Director Task, without chain-of-thought, the panel makes the egocentric error on 78\% of trials like children rather than adults; variation is substantial across models, and reasoning rescues several models. On the triangles, the panel under-attributes intention: its ToM profile sits more than three times closer to the high-functioning-autistic-adult (HF-ASD) mean than to the typical-development-adult (TD) mean, while Goal-Directed and Random stay near TD. No model is nearest TD on both tasks; the model that looks adult-like on the Director Task falls on the HF-ASD side on the triangles, and the most TD-like model on the triangles is child-like on the Director Task. We report group-level descriptions, not diagnostic labels for any model.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Multiagent Systems (cs.MA); Neurons and Cognition (q-bio.NC)
arXiv:2608.00261 [cs.CL]
(or arXiv:2608.00261v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.00261 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 215 words
Related coverage
AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work (July 2026)
AI Alignment Forum · Jul 31, 2026
In-Context Collapse in Vision-Language Models and How to Mitigate it?
arXiv · Aug 5, 2026
[AINews] High Taste Testers Ep. 1
Latent Space · Oct 17, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026