analiza aiqbench
AVI-Bench introduces a cognitively inspired evaluation framework that exposes significant gaps in the audio-visual intelligence of current Omni-MLLMs across perception, understanding, and reasoning stages. By incorporating a probe for primitive sensations using unfamiliar stimuli, the benchmark effectively tests model generalization beyond standard training distributions. This principled approach provides a necessary taxonomy and diagnostic tool to guide the development of more robust multimodal systems.
AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs
arXiv:2606.07643v1 Announce Type: cross Abstract: Recent advances in Omni-Multimodal Large Language Models (Omni-MLLMs) have enabled strong integration of vision, audio, and language. However, their audio-visual intelligence (AVI) remains insufficiently evaluated due to the lack of systematic and comprehensive benchmarks. We introduce AVI-Bench, a cognitively inspired benchmark that evaluates Omni-MLLMs across three stages, perception, understanding, and reasoning, through cross-modal tasks requiring joint audio-visual interpretation. This design enables fine-grained diagnosis of model capabilities and failure modes. To further assess robustness beyond familiar domains, we propose AVI-Bench-PriSe, an extension that probes models' primitive audio-visual sensation using unfamiliar, low-semantic stimuli, testing generalization beyond common training distributions. Extensive experiments on both open-source and closed-source models reveal substantial limitations in current Omni-MLLMs. Based on these findings, we present a four-level AVI taxonomy. Overall, AVI-Bench provides a principled evaluation framework to guide the development of more robust and generalizable AVI. Project website: https://fudancvl.github.io/AVI-Bench/
Abstract:Recent advances in Omni-Multimodal Large Language Models (Omni-MLLMs) have enabled strong integration of vision, audio, and language. However, their audio-visual intelligence (AVI) remains insufficiently evaluated due to the lack of systematic and comprehensive benchmarks. We introduce AVI-Bench, a cognitively inspired benchmark that evaluates Omni-MLLMs across three stages, perception, understanding, and reasoning, through cross-modal tasks requiring joint audio-visual interpretation. This design enables fine-grained diagnosis of model capabilities and failure modes. To further assess robustness beyond familiar domains, we propose AVI-Bench-PriSe, an extension that probes models' primitive audio-visual sensation using unfamiliar, low-semantic stimuli, testing generalization beyond common training distributions. Extensive experiments on both open-source and closed-source models reveal substantial limitations in current Omni-MLLMs. Based on these findings, we present a four-level AVI taxonomy. Overall, AVI-Bench provides a principled evaluation framework to guide the development of more robust and generalizable AVI. Project website: this https URL
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)
arXiv:2606.07643 [cs.CV]
(or arXiv:2606.07643v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.07643 arXiv-issued DOI via DataCite (pending registration)
From: Yaoting Wang Mr. [view email] [v1] Mon, 1 Jun 2026 19:12:09 UTC (7,111 KB)
Text integral sub licența deschisă a publicației · 189 cuvinte
Subiecte conexe
[AINews] Quasi-Riemann-Hypothesis: OpenAI publishes 722 math papers solving 90 of the top 500 open math problems; “the most significant moment” in >100 years of mathematics
Latent Space · 7 oct. 2026
[AINews] The Future of Latent Space
Latent Space · 25 sept. 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · 29 sept. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026