analiza aiqbench
This paper introduces SOCIAL CAPTION, a novel framework for evaluating the social understanding capabilities of multimodal large language models across three specific dimensions. By leveraging MLLM judges to automate assessment, the study provides critical insights into how architectural design and scale influence a model's ability to interpret human interactions. This work advances the field by establishing rigorous benchmarks essential for developing more socially aware AI systems.
Social Caption: Evaluating Social Understanding in Multimodal Models
arXiv:2601.14569v3 Announce Type: replace Abstract: Social understanding abilities are crucial for multimodal large language models (MLLMs) to interpret human social interactions. We introduce SOCIAL CAPTION, a framework grounded in interaction theory to evaluate social understanding abilities of MLLMs along three dimensions: Social Inference (SI), the ability to make accurate inferences about interactions; Holistic Social Analysis (HSA), the ability to generate comprehensive descriptions of interactions; Directed Social Analysis (DSA), the ability to generate relevant information from interactions. We analyze factors influencing model performance in social understanding, such as scale, architectural design, and spoken context. Experiments with MLLM judges demonstrate a path towards scaling automated evaluation of multimodal social understanding.
Abstract:Social understanding abilities are crucial for multimodal large language models (MLLMs) to interpret human social interactions. We introduce SOCIAL CAPTION, a framework grounded in interaction theory to evaluate social understanding abilities of MLLMs along three dimensions: Social Inference (SI), the ability to make accurate inferences about interactions; Holistic Social Analysis (HSA), the ability to generate comprehensive descriptions of interactions; Directed Social Analysis (DSA), the ability to generate relevant information from interactions. We analyze factors influencing model performance in social understanding, such as scale, architectural design, and spoken context. Experiments with MLLM judges demonstrate a path towards scaling automated evaluation of multimodal social understanding.
Accepted to Findings of EMNLP 2026. 26 pages, 10 figures
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2601.14569 [cs.CL]
(or arXiv:2601.14569v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2601.14569 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 209 cuvinte
Subiecte conexe
Social Caption: Evaluating Social Understanding in Multimodal Models
arXiv · 3 iun. 2026
Reading Between the Frames: Interpreting Implicit and Non-literal Meaning in Social Media Videos
arXiv · 6 aug. 2026
Modeling Complex Behaviors: Multi-Personality Composition and Dynamic Switching in Vision-Language Models
arXiv · 11 iun. 2026
Modeling Complex Behaviors: Multi-Personality Composition and Dynamic Switching in Vision-Language Models
arXiv · 10 iun. 2026