aiqbench analysis
This research introduces Intent Projection, a framework that decouples literal visual descriptions from pragmatic intent in Large Vision Language Models to better interpret sarcasm and memes. By applying orthogonal projections and contrastive rewards, the method effectively prevents surface-level details from contaminating responses, significantly outperforming open-source baselines on high-divergence posts. The approach offers a practical pathway for improving the nuance and accuracy of multimodal models in understanding complex social media contexts.
Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding
arXiv:2606.03604v1 Announce Type: new Abstract: When asked what a meme or sarcastic post means, Large Vision Language Models (LVLMs) tend to describe what the image shows rather than what the author is trying to communicate. Standard instruction tuning entangles a post's literal content with its pragmatic meaning, letting surface-level details contaminate the final response. We reframe meme understanding as a problem of literal-pragmatic decomposition and propose \textbf{Intent Projection}, a framework that separates the two signals at the representation, output, and objective levels within a single LVLM backbone. At the representation level, an orthogonal projection module removes dominant unimodal directions from the fused image-text representation, retaining only the pragmatic residual, while a surface-real affect classifier anchors the decoder with a discrete tag that names the polarity gap. At the output level, the model externalizes a structured reasoning chain, and at the objective level a contrastive reward explicitly penalizes answers that restate the literal description. Across six multimodal benchmarks, Intent Projection consistently outperforms open-source baselines and narrows the gap to proprietary models, with the largest gains on high-divergence posts where literal collapse is most damaging.
Abstract:When asked what a meme or sarcastic post means, Large Vision Language Models (LVLMs) tend to describe what the image shows rather than what the author is trying to communicate. Standard instruction tuning entangles a post's literal content with its pragmatic meaning, letting surface-level details contaminate the final response. We reframe meme understanding as a problem of literal-pragmatic decomposition and propose \textbf{Intent Projection}, a framework that separates the two signals at the representation, output, and objective levels within a single LVLM backbone. At the representation level, an orthogonal projection module removes dominant unimodal directions from the fused image-text representation, retaining only the pragmatic residual, while a surface-real affect classifier anchors the decoder with a discrete tag that names the polarity gap. At the output level, the model externalizes a structured reasoning chain, and at the objective level a contrastive reward explicitly penalizes answers that restate the literal description. Across six multimodal benchmarks, Intent Projection consistently outperforms open-source baselines and narrows the gap to proprietary models, with the largest gains on high-divergence posts where literal collapse is most damaging.
Computation and Language (cs.CL)
arXiv:2606.03604 [cs.CL]
(or arXiv:2606.03604v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.03604 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 273 words
Related coverage
Metadata-Aware Multi-Prompt Reasoning for Zero-Shot Accident Understanding
arXiv · Jun 11, 2026
MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning
arXiv · Jul 28, 2026
Interpreting Language Model Parameters
arXiv · Oct 17, 2026
Learning to Think Like a Cartoon Captionist: Incongruity-Resolution Supervision for Multimodal Humor Understanding
arXiv · Sep 12, 2026