analiza aiqbench
This research advances natural language processing by addressing the nuanced challenge of distinguishing between surface-level causal claims and counter-causal statements in news text. The authors employ soft-vote ensembling of RoBERTa models and synthetic data augmentation via LLMs to significantly improve detection accuracy and polarity labeling on the Countercausal News Corpus. These methodological improvements offer a robust framework for building AI systems that can accurately interpret complex causal relationships without being misled by linguistic ambiguity.
Shrome at Touch\'e: Soft-Vote Ensembling and Counter-Causal Augmentation for Causality Extraction
arXiv:2610.03268v1 Announce Type: new Abstract: Touch\'e 2026 extends causality extraction to counter-causal claims: news sentences whose surface form appears causal but whose meaning denies the causation, as in "It is falsely believed that X caused Y." A system that relies on surface cues such as "caused" or "led to" will accept such a sentence as causal and give it the wrong polarity. On the Countercausal News Corpus (CCNC), the task has three subtasks: deciding whether a sentence is causal (detection), locating its cause and effect spans (extraction), and labeling its polarity as procausal, counter-causal, or uncausal. We build one model per subtask. Detection is a fine-tuned classifier with a single cross-task rule that uses the extracted spans to remove false positives. For extraction, we ensemble three RoBERTa-large BILOU+CRF taggers by averaging their token-level scores before decoding, rather than voting on the spans each tagger produces. For polarity, where labeled counter-causal examples are scarcest, we add training sentences generated by a large language model prompted with nine patterns of counter-causal expression adapted from Hagen et al., keeping only those that pass automatic structural checks. On the held-out CCNC test set, the system reaches F1 0.869 on detection and macro-F1 0.817 on polarity, and in the organizers' final causal-only evaluation of extraction it scores granularity-adjusted F1 0.728, the highest extraction score among all submissions including the organizers' baseline. The development split is used only for component selection and the ablations reported in the paper.
Abstract:Touché 2026 extends causality extraction to counter-causal claims: news sentences whose surface form appears causal but whose meaning denies the causation, as in "It is falsely believed that X caused Y." A system that relies on surface cues such as "caused" or "led to" will accept such a sentence as causal and give it the wrong polarity. On the Countercausal News Corpus (CCNC), the task has three subtasks: deciding whether a sentence is causal (detection), locating its cause and effect spans (extraction), and labeling its polarity as procausal, counter-causal, or uncausal. We build one model per subtask. Detection is a fine-tuned classifier with a single cross-task rule that uses the extracted spans to remove false positives. For extraction, we ensemble three RoBERTa-large BILOU+CRF taggers by averaging their token-level scores before decoding, rather than voting on the spans each tagger produces. For polarity, where labeled counter-causal examples are scarcest, we add training sentences generated by a large language model prompted with nine patterns of counter-causal expression adapted from Hagen et al., keeping only those that pass automatic structural checks. On the held-out CCNC test set, the system reaches F1 0.869 on detection and macro-F1 0.817 on polarity, and in the organizers' final causal-only evaluation of extraction it scores granularity-adjusted F1 0.728, the highest extraction score among all submissions including the organizers' baseline. The development split is used only for component selection and the ablations reported in the paper.
16 pages, 5 figures, 10 tables. Both authors contributed equally. Working notes of Touché at CLEF 2026 (Conference and Labs of the Evaluation Forum), 21-24 September 2026, Jena, Germany
Computation and Language (cs.CL)
arXiv:2610.03268 [cs.CL]
(or arXiv:2610.03268v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.03268 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 285 cuvinte
Subiecte conexe
Game-Theoretic Disempowerment: Loss of Control to Agencyless AI
LessWrong · 27 sept. 2026
CauTion: Knowing When to Trust LLMs for Ensemble Causal Discovery
arXiv · 3 iun. 2026
DWT-Fusion: A Signal-Based Framework for Training-Free LLM-Generated Text Detection
arXiv · 27 iul. 2026
Game Design Creativity evals
LessWrong · 12 oct. 2026