analiza aiqbench
This article is clearly AI-centric: it's a diagnostic study of multimodal LLMs in video understanding. The primary subject is model reliability, so it passes the topic gate. Among the categories, it best fits AI Research, since it is an evaluation paper probing a systematic failure mode rather than a product, policy, or safety-deployment piece. Its findings are useful for readers building or evaluating video-capable models: MLLMs tend to pick plausible distractors instead of rejecting when no option is correct, the problem worsens with denser frame sampling and temporal reasoning, and chain-of-thought helps only partially. The results suggest that dedicated detection mechanisms, not just prompting, will be needed.
When No Answer Is Correct: Diagnosing Absent Answer Detection for MLLMs in Video Understanding
arXiv:2606.08239v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have made substantial advancements in video understanding, yet the reliability of their responses remains underexplored. This work presents a diagnostic study of absent answer detection for MLLMs in video understanding, where the correct answer is deliberately excluded from the candidate set and a reliable model is expected to recognize that no valid option exists. We evaluate the absent answer detection behavior under three settings: multiple-choice questions augmented with a ``None of the Above'' option, open-ended generation with a detection instruction, and standard evaluation without any guidance. Across a diverse set of models and benchmarks, we find that MLLMs overwhelmingly select plausible distractors rather than detecting the absent answer. This failure is more pronounced in temporal reasoning tasks and worsens with denser frame sampling. We further explore chain-of-thought prompting as a mitigation strategy and find that while it substantially improves detection rates, performance remains unsatisfactory, suggesting that prompting-based strategies alone are insufficient to fully address this limitation. These findings expose a systematic failure in absent answer detection and highlight the need for explicit detection mechanisms in multimodal systems.
Abstract:Multimodal large language models (MLLMs) have made substantial advancements in video understanding, yet the reliability of their responses remains underexplored. This work presents a diagnostic study of absent answer detection for MLLMs in video understanding, where the correct answer is deliberately excluded from the candidate set and a reliable model is expected to recognize that no valid option exists. We evaluate the absent answer detection behavior under three settings: multiple-choice questions augmented with a ``None of the Above'' option, open-ended generation with a detection instruction, and standard evaluation without any guidance. Across a diverse set of models and benchmarks, we find that MLLMs overwhelmingly select plausible distractors rather than detecting the absent answer. This failure is more pronounced in temporal reasoning tasks and worsens with denser frame sampling. We further explore chain-of-thought prompting as a mitigation strategy and find that while it substantially improves detection rates, performance remains unsatisfactory, suggesting that prompting-based strategies alone are insufficient to fully address this limitation. These findings expose a systematic failure in absent answer detection and highlight the need for explicit detection mechanisms in multimodal systems.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2606.08239 [cs.AI]
(or arXiv:2606.08239v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.08239 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 284 cuvinte
Subiecte conexe
OmniJudge or OmniBias? Diagnosing Multimodal Judges through Balanced, Decoupled Lenses
arXiv · 27 aug. 2026
OmniEdu: Open Foundation Models for Learning and Teaching
arXiv · 22 sept. 2026
North or South? Diagnosing LLMs' Understanding of North Korean Language, Society, and Knowledge
arXiv · 15 oct. 2026
Hallucinations in LLMs: A Lifecycle-Based Survey of Causes, Detection, Mitigation, and Prevention
arXiv · 28 aug. 2026