aiqbench analysis
This research introduces EmoEUS, a framework addressing modality-specific uncertainty in multimodal emotion recognition by dynamically weighting inputs based on learned variance estimates. The method explicitly supervises the alignment between predicted distributional distances and cluster centers to improve robustness against conflicting cues or missing signals. Experiments demonstrate that this approach consistently outperforms existing state-of-the-art methods on standard benchmarks like IEMOCAP and MELD.
EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation
arXiv:2607.18336v1 Announce Type: cross Abstract: Multimodal emotion recognition in conversation (MERC) can leverage multimodal and contextual cues to boost recognition performance. However, existing fusion approaches in MERC often ignore modality-specific uncertainty across utterances caused by conflicting cues, varying noise, and missing modality-specific signals. We propose EmoEUS, an explicit uncertainty supervision framework for MERC. EmoEUS performs uncertainty-aware multimodal fusion by dynamically weighting modalities using learned variance estimates. We also introduce an explicitly supervised loss that aligns each utterance's predicted variance with the distance between the utterance's distributional representation and its emotion- and modality-specific cluster center. Experiments on IEMOCAP and MELD show that EmoEUS consistently outperforms state-of-the-art methods.
Abstract:Multimodal emotion recognition in conversation (MERC) can leverage multimodal and contextual cues to boost recognition performance. However, existing fusion approaches in MERC often ignore modality-specific uncertainty across utterances caused by conflicting cues, varying noise, and missing modality-specific signals. We propose EmoEUS, an explicit uncertainty supervision framework for MERC. EmoEUS performs uncertainty-aware multimodal fusion by dynamically weighting modalities using learned variance estimates. We also introduce an explicitly supervised loss that aligns each utterance's predicted variance with the distance between the utterance's distributional representation and its emotion- and modality-specific cluster center. Experiments on IEMOCAP and MELD show that EmoEUS consistently outperforms state-of-the-art methods.
Multimedia (cs.MM); Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2607.18336 [cs.MM]
(or arXiv:2607.18336v1 [cs.MM] for this version)
https://doi.org/10.48550/arXiv.2607.18336 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 199 words
Related coverage
EII-SCL: Harnessing Emotional Inertia for Multimodal Emotion Recognition in Conversation
arXiv · Jul 21, 2026
C$^2$MOE: Consistency and Complementarity-guided Mixture of Experts for Incomplete Multimodal Emotion Learning
arXiv · Aug 6, 2026
The Continual Constitutional Convention: How language models revise the documents that govern them
AI Alignment Forum · Oct 13, 2026
SCoPE: Shift-Aware Speaker-Conditioned Priors for Emotion Recognition in Conversations
arXiv · Jul 25, 2026