aiqbench analysis
This is an experimental multi-agent LLM prototype that compares how 17 outlets frame the same news events, with sentiment, lean, loaded-language, and omission analyses. Its evaluation is preliminary: lean is checked against approximate reference ratings, and the "grounding" scores are the model's own judgments of support in the text, not verified accuracy. Readers should treat it as a research tool for studying framing, not a validated fact-checker.
Same Story, Different Lens: A Multi-Agent System for Cross-Outlet News Framing Analysis
arXiv:2610.35208v1 Announce Type: new Abstract: News outlets covering the same event differ in what they emphasize and omit. \textit{Same Story, Different Lens} is an experimental research prototype for comparing these choices. It collects world and politics coverage from 17 outlets, groups reports about the same event, and produces structured analyses of sentiment, political lean, framing, loaded language, possible omissions, and model-estimated proportions of factual claims and opinion. Loaded-language entries include quotations and neutral rewordings. Three model roles (agents) handle article analysis, assessment of source support, and cross-outlet briefing generation. Analyses are stored in a JSONL archive. The evaluation compares outlet-level lean estimates with approximate, predefined reference ratings and summarizes the Critic's model-assigned grounding scores. These measures provide preliminary diagnostics. A grounding score reflects the model's assessment of support in the supplied text and does not establish factual accuracy or constitute independent verification.
Abstract:News outlets covering the same event differ in what they emphasize and omit. \textit{Same Story, Different Lens} is an experimental research prototype for comparing these choices. It collects world and politics coverage from 17 outlets, groups reports about the same event, and produces structured analyses of sentiment, political lean, framing, loaded language, possible omissions, and model-estimated proportions of factual claims and opinion. Loaded-language entries include quotations and neutral rewordings. Three model roles (agents) handle article analysis, assessment of source support, and cross-outlet briefing generation. Analyses are stored in a JSONL archive. The evaluation compares outlet-level lean estimates with approximate, predefined reference ratings and summarizes the Critic's model-assigned grounding scores. These measures provide preliminary diagnostics. A grounding score reflects the model's assessment of support in the supplied text and does not establish factual accuracy or constitute independent verification.
The projects source code is available in the GitHub repository. Link: this https URL
Artificial Intelligence (cs.AI)
arXiv:2610.35208 [cs.AI]
(or arXiv:2610.35208v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.35208 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 245 words