aiqbench analysis
This study uses face pareidolia as a diagnostic tool to reveal how different vision architectures handle ambiguous visual evidence, distinguishing between semantic overactivation in VLMs and conservative suppression in detection models. The findings demonstrate that uncertainty metrics alone are insufficient for assessing robustness, as low confidence can mask either safe abstention or extreme misinterpretation depending on the underlying representation. By establishing face pareidolia as a source of hard negatives, this work offers critical insights into improving semantic reliability across diverse computer vision systems.
When Visual Evidence is Ambiguous: Pareidolia as a Diagnostic Probe for Vision Models
arXiv:2603.03989v2 Announce Type: replace-cross Abstract: When visual evidence is ambiguous, vision models must decide how to interpret face-like patterns. Face pareidolia, the perception of faces in non-face objects, provides a controlled probe of such decisions. We introduce a diagnostic framework that analyzes detection, localization, uncertainty and bias across class, difficulty and emotion. We evaluate six models spanning four representational regimes: vision-language models (VLMs; CLIP-B/32, CLIP-L/14, LLaVA-1.5-7B), pure vision classification (ViT), object detection (YOLOv8), and face detection (RetinaFace). Our results reveal that uncertainty and bias are decoupled: low uncertainty can signal either safe suppression, as in detectors, or extreme over-interpretation, as in VLMs. VLMs exhibit semantic overactivation, systematically interpreting ambiguous non-human regions as Human, with LLaVA over-calling on 73% of non-human pareidolic images, especially for negative emotions. ViT instead follows an uncertainty-as-abstention strategy, remaining diffuse yet largely unbiased. Detection-based models achieve low bias through conservative priors that suppress pareidolia responses even when localization is controlled. Together, these results show that behavior under ambiguity is governed more by representation than thresholds, establishing face pareidolia as a diagnostic of semantic robustness and a source of ambiguity-aware hard negatives for vision models. Code will be released upon publication.
Abstract:When visual evidence is ambiguous, vision models must decide how to interpret face-like patterns. Face pareidolia, the perception of faces in non-face objects, provides a controlled probe of such decisions. We introduce a diagnostic framework that analyzes detection, localization, uncertainty and bias across class, difficulty and emotion. We evaluate six models spanning four representational regimes: vision-language models (VLMs; CLIP-B/32, CLIP-L/14, LLaVA-1.5-7B), pure vision classification (ViT), object detection (YOLOv8), and face detection (RetinaFace). Our results reveal that uncertainty and bias are decoupled: low uncertainty can signal either safe suppression, as in detectors, or extreme over-interpretation, as in VLMs. VLMs exhibit semantic overactivation, systematically interpreting ambiguous non-human regions as Human, with LLaVA over-calling on 73% of non-human pareidolic images, especially for negative emotions. ViT instead follows an uncertainty-as-abstention strategy, remaining diffuse yet largely unbiased. Detection-based models achieve low bias through conservative priors that suppress pareidolia responses even when localization is controlled. Together, these results show that behavior under ambiguity is governed more by representation than thresholds, establishing face pareidolia as a diagnostic of semantic robustness and a source of ambiguity-aware hard negatives for vision models. Code will be released upon publication.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
arXiv:2603.03989 [cs.CV]
(or arXiv:2603.03989v2 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2603.03989 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 287 words
Related coverage
VIDA: A Dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
arXiv · Oct 2, 2026
MVGraph: Benchmarking LVLMs on Evidence Composition across Multi-View Visual Graph
arXiv · Oct 12, 2026
EviSafe: Evidence-Grounded Safety Evaluation for Vision-Language Models
arXiv · Aug 25, 2026
MagiC: Evaluating Multimodal Cognition Toward Grounded Visual Reasoning
arXiv · Oct 16, 2026