aiqbench analysis
FriendBench introduces a novel benchmark evaluating multimodal models' ability to infer social familiarity from brief dyadic interactions, revealing that top AI systems often rely on biased priors rather than genuine behavioral discrimination. The study highlights a critical gap where humans leverage visible cues effectively while current LLMs struggle with this specific type of context-dependent reasoning without explicit training data. This work advances the field by providing open stimuli and predictions to better understand how multimodal models interpret non-verbal social dynamics compared to human baselines.
FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models
arXiv:2607.29602v1 Announce Type: new Abstract: Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a benchmark for inferring whether two people are already familiar or are meeting as strangers, from a 20-second clip of a dyadic ice-breaker conversation. Every pair answers the same type of prompt, so only the manner of interaction can reveal the answer. Across text, audio, and video, we compare 26 models from seven companies against matched human panels over 96 balanced dyads. The best model and the human crowd are statistically indistinguishable on accuracy in every modality, but reach it differently: humans stay balanced across the two answers, while the strongest models lean toward "stranger"---a difference in effective prior, not discrimination. Richer channels help both unequally, and only humans gain from visible behavior on top of speech. We release the stimuli, human ratings, and model predictions.
Abstract:Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a benchmark for inferring whether two people are already familiar or are meeting as strangers, from a 20-second clip of a dyadic ice-breaker conversation. Every pair answers the same type of prompt, so only the manner of interaction can reveal the answer. Across text, audio, and video, we compare 26 models from seven companies against matched human panels over 96 balanced dyads. The best model and the human crowd are statistically indistinguishable on accuracy in every modality, but reach it differently: humans stay balanced across the two answers, while the strongest models lean toward "stranger"---a difference in effective prior, not discrimination. Richer channels help both unequally, and only humans gain from visible behavior on top of speech. We release the stimuli, human ratings, and model predictions.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Human-Computer Interaction (cs.HC)
arXiv:2607.29602 [cs.CL]
(or arXiv:2607.29602v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2607.29602 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 171 words
Related coverage
[AINews] OpenAI reports Navier-Stokes singularity find in 88 hours using Astra-next, roughly 10,000 agents and 130B tokens (>$40M), a contender for second ever Millennium Prize awarded
Latent Space · Sep 9, 2026
[AINews] not much happened today
Latent Space · Sep 18, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026