analiza aiqbench
This research paper introduces Unified Multimodal Uncertain Inference (UMUI), a novel framework enabling large language models to perform fine-grained probabilistic reasoning across text, audio, and video modalities. By leveraging the CLUE method for calibrated latent uncertainty estimation, the study demonstrates that a 3B-parameter model can match or exceed larger baselines in generating reliable probability estimates for complex multimodal hypotheses. This advancement addresses a critical gap in current AI capabilities regarding cross-modal confidence calibration beyond simple binary entailment tasks.
Unified Multimodal Uncertain Inference
arXiv:2604.08701v3 Announce Type: replace-cross Abstract: We introduce Unified Multimodal Uncertain Inference (UMUI), a multimodal inference task spanning text, audio, and video, where models must produce calibrated probability estimates of hypotheses conditioned on a premise in any modality or combination. While uncertain inference has been explored in text, extension to other modalities has been limited to single-modality binary entailment judgments, leaving no framework for fine-grained probabilistic reasoning in or across other modalities. To address this, we curate a human-annotated evaluation set with scalar probability judgments across audio, visual, and audiovisual settings, and additionally evaluate on existing text and audio benchmarks. We introduce CLUE (Calibrated Latent Uncertainty Estimation), which combines self-consistent teacher calibration and distribution-based confidence probing to produce calibrated predictions. We demonstrate that our 3B-parameter model achieves equivalent or stronger performance than zero-shot baselines up to 32B parameters across all modalities.
Abstract:We introduce Unified Multimodal Uncertain Inference (UMUI), a multimodal inference task spanning text, audio, and video, where models must produce calibrated probability estimates of hypotheses conditioned on a premise in any modality or combination. While uncertain inference has been explored in text, extension to other modalities has been limited to single-modality binary entailment judgments, leaving no framework for fine-grained probabilistic reasoning in or across other modalities. To address this, we curate a human-annotated evaluation set with scalar probability judgments across audio, visual, and audiovisual settings, and additionally evaluate on existing text and audio benchmarks. We introduce CLUE (Calibrated Latent Uncertainty Estimation), which combines self-consistent teacher calibration and distribution-based confidence probing to produce calibrated predictions. We demonstrate that our 3B-parameter model achieves equivalent or stronger performance than zero-shot baselines up to 32B parameters across all modalities.
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2604.08701 [cs.CV]
(or arXiv:2604.08701v3 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2604.08701 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 233 cuvinte
Subiecte conexe
[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · 17 sept. 2026
NVIDIA Nemotron 3 Nano Omni Powers Multimodal Agent Reasoning in a Single Efficient Open Model
NVIDIA · 28 apr. 2026
UniT: Unified Multimodal Chain-of-Thought Test-time Scaling
arXiv · 16 iun. 2026
How to Build In-Vehicle AI Agents with NVIDIA: From Cloud to Car
NVIDIA · 5 mai 2026