analiza aiqbench
The ArGuard shared task presents significant new benchmarks for detecting harmful content in Arabic, specifically addressing the challenges of multimodal hate speech in memes and safety evaluation for Large Language Model prompts. With 27 teams submitting papers using models like AraBERT and Qwen3-VL, this research offers critical insights into mitigating bias and improving safety guardrails for non-English AI systems.
ArGuard Shared Task: Harmful Content Detection in Arabic Memes and LLM Prompts
arXiv:2609.29349v1 Announce Type: new Abstract: ArGuard is a shared task on harmful content detection in Arabic memes and LLM prompts. It includes two tracks: Track A focuses on multimodal hate detection in Arabic memes, while Track B addresses harmful prompt detection for Arabic LLM safety evaluation. In total, 58 teams registered, 35 participated in the final evaluation, and 27 submitted system-description papers. Participating teams explored models such as AraBERT, Jais, and Qwen3-VL. The best systems achieved macro-F1 scores of 0.823 on A1, 0.419 on A2, 0.984 on B1, and 0.790 on B2. Fine-grained meme classification in A2 was the most challenging setting, partly due to sparse labels and train-test distribution shifts.
Abstract:ArGuard is a shared task on harmful content detection in Arabic memes and LLM prompts. It includes two tracks: Track A focuses on multimodal hate detection in Arabic memes, while Track B addresses harmful prompt detection for Arabic LLM safety evaluation. In total, 58 teams registered, 35 participated in the final evaluation, and 27 submitted system-description papers. Participating teams explored models such as AraBERT, Jais, and Qwen3-VL. The best systems achieved macro-F1 scores of 0.823 on A1, 0.419 on A2, 0.984 on B1, and 0.790 on B2. Fine-grained meme classification in A2 was the most challenging setting, partly due to sparse labels and train-test distribution shifts.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2609.29349 [cs.CL]
(or arXiv:2609.29349v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.29349 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 204 cuvinte
Subiecte conexe
Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memes
arXiv · 17 iun. 2026
From Recognition to Reasoning: Advancing Multimodal Harmful Meme Detection via Chain-of-Thought Alignment
arXiv · 25 aug. 2026
AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes
arXiv · 31 iul. 2026
Decodable but Misrouted: Sparse Features Uncover a Readout Gap in Vision-Language Models for Harmful Meme Detection
arXiv · 17 sept. 2026