Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factors

Title:Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factors

Abstract:Visual persuasion uses images to shape cognition, emotion, and behavior, with its effects depending on both visual attributes and semantic context. Despite recent progress, it remains unclear whether Vision-Language Models (VLMs) understand visual persuasiveness. This motivates us to ask: can VLMs assess whether an image persuasively supports an intended message, which visual factors shape this judgment, and do they align with human judgments? Through empirical analyses on image-message pairs where human raters consistently agree on the persuasiveness judgment, we show that VLMs exhibit a recall-oriented bias: they over-predict images as persuasive while achieving high recall. We introduce Visual Persuasive Factors (VPFs), a taxonomy informed by cognitive psychology for quantifying visual cues that shape persuasive judgments. Our factor-level analysis reveals that VPFs distinguish human persuasiveness judgments, whereas VLMs only partially reproduce these patterns, often generating false positives by treating persuasion-relevant cues as sufficient evidence. Building on this insight, we evaluate VPF-guided interventions and find that properly framed VPF knowledge can improve performance, but merely specifying visual cues or adding step-by-step reasoning is insufficient. By analyzing model rationales at the level of functional reasoning steps, we further identify a central bottleneck in connecting object identification to semantic message alignment.

Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)

arXiv:2511.17036 [cs.CL]

(or arXiv:2511.17036v2 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2511.17036 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.