analiza aiqbench
MemeLens introduces a unified, multilingual Vision-Language Model designed to handle diverse tasks like hate speech detection and sentiment analysis within memes. By consolidating 38 datasets into a shared taxonomy, the research demonstrates that robust multimodal understanding requires unified training rather than over-specialization on individual sets. This work advances the field of computer vision and language modeling by addressing cross-domain generalization in culturally complex visual-textual interactions.
MemeLens: Multilingual Multitask VLMs for Memes
arXiv:2601.12539v4 Announce Type: replace Abstract: Memes are a dominant medium for online communication and manipulation because meaning emerges from interactions between embedded text, imagery, and cultural context. Existing meme research is distributed across tasks (e.g., \textit{hate, misogyny, propaganda, sentiment, humour}) and languages, which limits cross-domain generalization. To address this gap, we propose \textsc{MemeLens}, a unified multilingual, multitask explanation-enhanced Vision-Language Model (VLM) for meme understanding. We consolidate $38$ public meme datasets, filter and map dataset-specific labels into a shared taxonomy of $20$ tasks spanning harm, targets, figurative/pragmatic intent, and affect. We present a comprehensive empirical analysis across modeling paradigms, task categories, and datasets. Our findings suggest that robust meme understanding requires multimodal training, varies substantially across semantic categories, and remains sensitive to over-specialization when models are fine-tuned on individual datasets rather than trained in a unified setting. We make the experimental resources (https://github.com/MohamedBayan/MemeLens), model (https://huggingface.co/QCRI/MemeLens-VLM) and datasets (https://huggingface.co/datasets/QCRI/MemeLens) publicly available to the community.
Abstract:Memes are a dominant medium for online communication and manipulation because meaning emerges from interactions between embedded text, imagery, and cultural context. Existing meme research is distributed across tasks (e.g., \textit{hate, misogyny, propaganda, sentiment, humour}) and languages, which limits cross-domain generalization. To address this gap, we propose \textsc{MemeLens}, a unified multilingual, multitask explanation-enhanced Vision-Language Model (VLM) for meme understanding. We consolidate $38$ public meme datasets, filter and map dataset-specific labels into a shared taxonomy of $20$ tasks spanning harm, targets, figurative/pragmatic intent, and affect. We present a comprehensive empirical analysis across modeling paradigms, task categories, and datasets. Our findings suggest that robust meme understanding requires multimodal training, varies substantially across semantic categories, and remains sensitive to over-specialization when models are fine-tuned on individual datasets rather than trained in a unified setting. We make the experimental resources (this https URL), model (this https URL) and datasets (this https URL) publicly available to the community.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2601.12539 [cs.AI]
(or arXiv:2601.12539v4 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2601.12539 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 249 cuvinte
Subiecte conexe
MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models
arXiv · 3 sept. 2026
Multilingual Training and Evaluation Resources for Vision-Language Models
arXiv · 9 iun. 2026
MLingualFC: Evaluating Jailbreak Vulnerabilities in Multilingual Vision-Language Models
arXiv · 9 iun. 2026
PEST: Parameter Efficient Steering of Blackbox VLMs via Agentic Few-shot Alignment for Hateful Meme Moderation
arXiv · 30 sept. 2026