aiqbench analysis
MACD addresses hallucinations in Video-LLMs by leveraging model feedback to generate targeted object-level counterfactuals, outperforming random perturbation methods in decoding evidence-grounded tokens. This approach effectively mitigates errors involving small, occluded, or co-occurring objects while maintaining task accuracy across diverse models like Qwen and InternVL. The technique represents a significant step toward more reliable visual-language reasoning by aligning inference strategies with specific model weaknesses.
MACD: Model-Aware Contrastive Decoding via Counterfactual Data
arXiv:2602.01740v3 Announce Type: replace-cross Abstract: Video language models (Video-LLMs) are prone to hallucinations, generating plausible but ungrounded content when visual evidence is weak, ambiguous, or biased. Existing methods, such as contrastive decoding (CD), rely on random perturbations to construct contrastive data for hallucination mitigation, but often fail to target the visual cues that drive hallucination or align with model weaknesses. We propose Model-Aware Counterfactual Data based Contrastive Decoding (MACD), an inference strategy that combines model-guided counterfactual construction with contrastive decoding. MACD uses the Video-LLM's own feedback to identify object regions most responsible for hallucination, generating targeted object-level counterfactual inputs rather than arbitrary frame or temporal modifications. These counterfactual inputs are integrated into CD to enforce evidence-grounded token selection during decoding. Experiments on EventHallusion, MVBench, Perception-test, and Video-MME show that MACD consistently reduces hallucination while maintaining or improving task accuracy across diverse Video-LLMs, including Qwen and InternVL, with especially strong gains in scenarios involving small, occluded, or co-occurring objects.
Abstract:Video language models (Video-LLMs) are prone to hallucinations, generating plausible but ungrounded content when visual evidence is weak, ambiguous, or biased. Existing methods, such as contrastive decoding (CD), rely on random perturbations to construct contrastive data for hallucination mitigation, but often fail to target the visual cues that drive hallucination or align with model weaknesses. We propose Model-Aware Counterfactual Data based Contrastive Decoding (MACD), an inference strategy that combines model-guided counterfactual construction with contrastive decoding. MACD uses the Video-LLM's own feedback to identify object regions most responsible for hallucination, generating targeted object-level counterfactual inputs rather than arbitrary frame or temporal modifications. These counterfactual inputs are integrated into CD to enforce evidence-grounded token selection during decoding. Experiments on EventHallusion, MVBench, Perception-test, and Video-MME show that MACD consistently reduces hallucination while maintaining or improving task accuracy across diverse Video-LLMs, including Qwen and InternVL, with especially strong gains in scenarios involving small, occluded, or co-occurring objects.
Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2602.01740 [cs.AI]
(or arXiv:2602.01740v3 [cs.AI] for this version)
Full text under the publisher's open licence · 173 words
Related coverage
Model-Aware Data Selection from In-and-Out Information Interplay
arXiv · Sep 29, 2026
[AINews] High Taste Testers Ep. 1
Latent Space · Oct 17, 2026
Why do models task game?
AI Alignment Forum · Aug 6, 2026
ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Models
arXiv · Sep 30, 2026