analiza aiqbench
This research establishes a principled framework for assessing the robustness of Visual Language Model hallucinations by quantifying the sample complexity required to detect instability under counterfactual perturbations. By integrating circuit discovery techniques with causal influence metrics, the study identifies specific model components driving unreliable predictions and derives empirical bounds on the minimum counterfactual samples needed for reliable detection. These findings offer a rigorous methodological advance for evaluating the stability and grounding of VLM outputs in safety-critical applications.
How Many Counterfactuals Does It Take? Probing VLM Hallucinations Through Circuits and Causal Effects
arXiv:2606.08777v1 Announce Type: new Abstract: Visual Language Models (VLMs) are known to produce hallucinated predictions that are not grounded in visual evidence, yet existing approaches lack a principled understanding of how robust such predictions are under counterfactual perturbations. In this work, we study the sample complexity of counterfactual robustness for hallucinated outputs in VLMs. We define a causal influence metric based on log-probability differences between factual, counterfactual, and activation-patched runs, and use it to characterize the stability of hallucinated predictions. By leveraging circuit discovery techniques (CD-T), we identify model components responsible for these predictions and track their activation differences across counterfactual samples. We then derive empirical bounds on the minimum number of counterfactual samples m required to reliably detect instability in hallucinated outputs, using concentration inequalities and variance estimates of the causal influence distribution.
Title:How Many Counterfactuals Does It Take? Probing VLM Hallucinations Through Circuits and Causal Effects
Abstract:Visual Language Models (VLMs) are known to produce hallucinated predictions that are not grounded in visual evidence, yet existing approaches lack a principled understanding of how robust such predictions are under counterfactual perturbations. In this work, we study the sample complexity of counterfactual robustness for hallucinated outputs in VLMs. We define a causal influence metric based on log-probability differences between factual, counterfactual, and activation-patched runs, and use it to characterize the stability of hallucinated predictions. By leveraging circuit discovery techniques (CD-T), we identify model components responsible for these predictions and track their activation differences across counterfactual samples. We then derive empirical bounds on the minimum number of counterfactual samples m required to reliably detect instability in hallucinated outputs, using concentration inequalities and variance estimates of the causal influence distribution.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.08777 [cs.LG]
(or arXiv:2606.08777v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.08777 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 240 cuvinte
Subiecte conexe
Multi-agent Undercover Gaming: Hallucination Removal via Counterfactual Test for Multimodal Reasoning
arXiv · 15 oct. 2026
MACD: Model-Aware Contrastive Decoding via Counterfactual Data
arXiv · 8 iun. 2026
Explainable AML Triage with LLMs: Evidence Retrieval and Counterfactual Checks
arXiv · 9 iun. 2026
Causal-Counterfactual RAG: The Integration of Causal-Counterfactual Reasoning into RAG
arXiv · 4 sept. 2026