analiza aiqbench
The paper shows that fine-tuned models plateau at about 0.67 F1 for enthymeme detection on vaccination and immigration tweets, whatever the data size or features. Its more useful contribution is methodological: it evaluates predictions against the full distribution of annotator votes and finds that models recognize enthymeme-free tweets well but rarely commit on tweets that contain one. For AI readers, the takeaway is that aggregate accuracy can hide systematic blind spots in implicit reasoning, a caveat that matters for moderation systems trained on majority labels.
Interpretive Variation and Discourse Structure in Detecting Enthymemes in Controversial Political Discourse
arXiv:2610.30987v1 Announce Type: new Abstract: Enthymemes are arguments that leave a premise or conclusion unstated, and they are a common persuasion device on social media. Because readers must fill in the missing part themselves, they often disagree on whether a text contains one. This paper asks how to detect and evaluate enthymemes when such disagreement is part of the task. On tweets about vaccination and immigration, each labeled by five annotators, F1 stalls around 0.67 for every fine-tuned model we train, regardless of data size, added features or label type. We therefore ask how these models should be evaluated instead, and propose metrics that compare predictions with the full distribution of annotator votes, together with a model that predicts this distribution directly and captures disagreement better at no cost in F1. We then ask what role discourse structure plays. Models learn to rely on explicit argumentative structure: they reliably recognize tweets without an enthymeme but rarely commit on tweets that contain one. Both the models and a discourse parser miss arguments whose links are left implicit and must be inferred by the reader. Finally, we discuss what these blind spots mean for content moderation systems trained on majority labels.
Abstract:Enthymemes are arguments that leave a premise or conclusion unstated, and they are a common persuasion device on social media. Because readers must fill in the missing part themselves, they often disagree on whether a text contains one. This paper asks how to detect and evaluate enthymemes when such disagreement is part of the task. On tweets about vaccination and immigration, each labeled by five annotators, F1 stalls around 0.67 for every fine-tuned model we train, regardless of data size, added features or label type. We therefore ask how these models should be evaluated instead, and propose metrics that compare predictions with the full distribution of annotator votes, together with a model that predicts this distribution directly and captures disagreement better at no cost in F1. We then ask what role discourse structure plays. Models learn to rely on explicit argumentative structure: they reliably recognize tweets without an enthymeme but rarely commit on tweets that contain one. Both the models and a discourse parser miss arguments whose links are left implicit and must be inferred by the reader. Finally, we discuss what these blind spots mean for content moderation systems trained on majority labels.
Computation and Language (cs.CL)
arXiv:2610.30987 [cs.CL]
(or arXiv:2610.30987v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.30987 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 289 cuvinte
Subiecte conexe

Niels Bohr
Quantum Zeitgeist · 22 iul. 2026

Presentation: Designing AI Platforms for Reliability: Tools for Certainty, Agents for Discovery
InfoQ · 27 mai 2026
By Their Fruits You Will Know Them: Comparing Formalizations of Law by the Decisions They Encode
arXiv · 18 sept. 2026
A Resource for Enthymeme Detection in Controversial Political Discourse
arXiv · 11 iun. 2026