aiqbench analysis
This research demonstrates effective fact-checking system design by leveraging a tiny Llama-3.2 model strictly as a citation validator rather than a prose generator, significantly improving metric scores through conservative scoring and selective emission policies. The study highlights critical lessons on judge calibration and the limitations of auxiliary signals when constructing reliable generated content for verification tasks.
UTS at CheckThat! 2026: Cite-Frame Engineering for Generated Fact-Checking Articles
arXiv:2608.24466v1 Announce Type: cross Abstract: CheckThat! 2026 Task 3 asks systems to generate fact-checking articles, graded by an unweighted mean of four sub-metrics (M4). Our UTS submission placed 2nd of 11 teams (M4 = 0.484). The shipped system is a deterministic stub drafter wrapped by two single-lever interventions: a domain-attribution cite frame (HostCite) and a shadow-validated anchor picker (ShadowVal) that use Llama-3.2:1B only as a per-cite validator, never as a body-prose generator. The stack lifts M4 by +0.027 over the stub on the WatClaimCheck validation split, beats the field on entailment and coverage, and follows two design rules our ablation matrix made unambiguous. Scorer conservatism: credit only tokens the references entail - templates pay; LLM prose, reviewer names, and raw evidence all fail. Auxiliary anchor signals are miscalibrated against the Llama judge: every anchor proxy we tried (cross-encoder, length, lead position) picks anchors the judge rejects - gate on the judge itself. The remaining +0.062 gap to the winner sits on citation precision/recall (0.299 vs 0.671), consistent with a selective-emission policy that drops low-confidence cites.
Title:UTS at CheckThat! 2026: Cite-Frame Engineering for Generated Fact-Checking Articles
Abstract:CheckThat! 2026 Task 3 asks systems to generate fact-checking articles, graded by an unweighted mean of four sub-metrics (M4). Our UTS submission placed 2nd of 11 teams (M4 = 0.484). The shipped system is a deterministic stub drafter wrapped by two single-lever interventions: a domain-attribution cite frame (HostCite) and a shadow-validated anchor picker (ShadowVal) that use Llama-3.2:1B only as a per-cite validator, never as a body-prose generator. The stack lifts M4 by +0.027 over the stub on the WatClaimCheck validation split, beats the field on entailment and coverage, and follows two design rules our ablation matrix made unambiguous. Scorer conservatism: credit only tokens the references entail - templates pay; LLM prose, reviewer names, and raw evidence all fail. Auxiliary anchor signals are miscalibrated against the Llama judge: every anchor proxy we tried (cross-encoder, length, lead position) picks anchors the judge rejects - gate on the judge itself. The remaining +0.062 gap to the winner sits on citation precision/recall (0.299 vs 0.671), consistent with a selective-emission policy that drops low-confidence cites.
CLEF2026, CheckThat!2026
Digital Libraries (cs.DL); Computation and Language (cs.CL)
arXiv:2608.24466 [cs.DL]
(or arXiv:2608.24466v1 [cs.DL] for this version)
https://doi.org/10.48550/arXiv.2608.24466 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 280 words
Related coverage
Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
AI Alignment Forum · Aug 3, 2026
LiveLaw Supreme Court Half-Yearly Digest: January - June 2026 [Part 2]
livelaw.in · Sep 13, 2026
Does MiniMax Agent Actually Make Work Easier?
KDnuggets · Aug 3, 2026
[AINews] Meta Muse Wonder & Wonder Full: MSL’s Claude 5.5-tier Frontier Model
Latent Space · Oct 13, 2026