analiza aiqbench
This paper proposes a behavioral benchmark that scores 45 vision models against published human perception data on four Gestalt grouping tasks, without requiring new user studies. Its key finding is that alignment with human perceptual organization diverges from standard accuracy, and some closed foundation models score noticeably lower on alignment than their benchmark results suggest. For readers, the practical point is that accuracy alone may not show whether a model's visual organization matches human interpretation, which matters for any pipeline that depends on models reading charts or visual data the way people do.
More Accurate, Less Human: Gestalt Grouping in Vision Models
arXiv:2608.10195v2 Announce Type: replace-cross Abstract: Human vision organizes what it sees into wholes: same-colored points group into series, similar marks cohere into categories, and shapes complete into recognizable objects. These are the Gestalt operations that visualization design builds on. Whether vision models organize visual content this way has not been systematically tested. We introduce a behavioral battery that scores models against human data from prior perception studies on four grouping tasks: mark-color odd-one-out, color-series counting, silhouette recognition, and object odd-one-out. We apply it to 45 models across five training families: supervised, self-supervised, and contrastive vision-language encoders, open-weight VLMs, and closed foundation models. The battery reveals that agreement with human responses captures aspects of perceptual organization that conventional performance metrics fail to distinguish, with several closed models exhibiting substantially lower alignment than their benchmark accuracy would suggest. Scoring against published perception data therefore gives visualization research a reusable yardstick, requiring no new user study, for auditing whether the models now entering visualization pipelines organize what they see the way their human audience does.
Abstract:Human vision organizes what it sees into wholes: same-colored points group into series, similar marks cohere into categories, and shapes complete into recognizable objects. These are the Gestalt operations that visualization design builds on. Whether vision models organize visual content this way has not been systematically tested. We introduce a behavioral battery that scores models against human data from prior perception studies on four grouping tasks: mark-color odd-one-out, color-series counting, silhouette recognition, and object odd-one-out. We apply it to 45 models across five training families: supervised, self-supervised, and contrastive vision-language encoders, open-weight VLMs, and closed foundation models. The battery reveals that agreement with human responses captures aspects of perceptual organization that conventional performance metrics fail to distinguish, with several closed models exhibiting substantially lower alignment than their benchmark accuracy would suggest. Scoring against published perception data therefore gives visualization research a reusable yardstick, requiring no new user study, for auditing whether the models now entering visualization pipelines organize what they see the way their human audience does.
Accepted to VISxVision 2026, a workshop at IEEE VIS 2026. Includes appendix with per-task stimuli, metric derivations, and full per-model results
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
H.5.2; I.2.10; I.4.8
arXiv:2608.10195 [cs.CV]
(or arXiv:2608.10195v2 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.10195 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 289 cuvinte
Subiecte conexe
AI Alignment for Digital Democracy: Fairer Voters Representation against LLM Biases
arXiv · 27 oct. 2026
Lattice and PT symmetries in tensor-network renormalization group: Case study of a hard-square lattice gas model
arXiv · 24 iul. 2026
From Self to Other: Evaluating Demographic Perspective-Taking in LLM Hate Speech Annotation
arXiv · 5 iun. 2026
Measuring Consensus in AI Agent Networks
arXiv · 21 oct. 2026