aiqbench analysis
This research demonstrates that foundation models can significantly improve content moderation performance by comparing instruction-driven reasoning against example-driven learning paradigms. Using a new benchmark of 4,000 real-world posts from Bluesky, the study finds that both approaches substantially outperform current deployed systems, offering scalable paths for reliable policy operationalization.
Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization
arXiv:2609.10410v1 Announce Type: cross Abstract: The growing complexity of content moderation policies presents a critical challenge for their consistent operationalization. While foundation models possess the basic capabilities needed to confront this challenge, whether they can reliably moderate online content remains an unanswered question. In this paper, we systematically compare two competing paradigms for Vision-Language Model (VLM) guidance: an instruction-driven approach where models reason from policy precepts, and an example-driven approach where they generalize from prior precedents. We ground this investigation in ModerationBench, a new benchmark of 4,000 manually annotated, in-the-wild posts from the Bluesky platform. Our experiments reveal that foundation models can substantially outperform Bluesky's deployed moderation system, nearly tripling its $F_1$ score (0.60 vs. 0.22) on Random Posts in the benchmark, with both instruction- and example-driven paradigms achieving comparable peak effectiveness. Our findings thus chart a path toward reliable and adaptable policy operationalization at scale.
Title:Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization
Abstract:The growing complexity of content moderation policies presents a critical challenge for their consistent operationalization. While foundation models possess the basic capabilities needed to confront this challenge, whether they can reliably moderate online content remains an unanswered question. In this paper, we systematically compare two competing paradigms for Vision-Language Model (VLM) guidance: an instruction-driven approach where models reason from policy precepts, and an example-driven approach where they generalize from prior precedents. We ground this investigation in ModerationBench, a new benchmark of 4,000 manually annotated, in-the-wild posts from the Bluesky platform. Our experiments reveal that foundation models can substantially outperform Bluesky's deployed moderation system, nearly tripling its $F_1$ score (0.60 vs. 0.22) on Random Posts in the benchmark, with both instruction- and example-driven paradigms achieving comparable peak effectiveness. Our findings thus chart a path toward reliable and adaptable policy operationalization at scale.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
arXiv:2609.10410 [cs.CL]
(or arXiv:2609.10410v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.10410 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 255 words
Related coverage
Links #5: 2026/07
LessWrong · Jul 31, 2026
Links #3: 2026/06 Part 1
LessWrong · Jun 15, 2026
The basic argument for making AI better at moral philosophy
LessWrong · Oct 14, 2026
Building trust in AI agents: A production-ready evaluation framework with Amazon Bedrock AgentCore
AWS · Oct 16, 2026