aiqbench analysis
This study reveals a critical flaw in relying solely on high annotation reproducibility when using LLMs as measurement tools, showing they can fail to capture the intended underlying constructs. The findings highlight systematic divergences between text-based analysis and survey responses across different stakeholder groups, with implications for how AI safety concerns are quantified in policy contexts. Researchers must now distinguish between model consistency and actual construct validity before deploying LLMs for social science or regulatory evaluation.
Reproducibility is not construct validity: LLM measurement of institutionally situated communication
arXiv:2609.19866v1 Announce Type: new Abstract: High annotation reproducibility does not necessarily imply that an LLM-inferred measure captures the construct it is intended to measure. We test this distinction using a dataset from the European Commission's AI Act consultation, linking structured survey responses to free-text consultation submissions from the same stakeholders. LLM annotations of consultation submissions are highly reproducible (intraclass correlations > 0.99), yet show limited convergence with survey-reported measures of the nominal construct they were intended to approximate. Divergence between survey-and LLM-inferred text-based measures varies systematically across stakeholder groups: business associations express greater concern about AI risks in text-based consultations than in survey responses ({\=g} = +1.0), whereas public authorities and several nonbusiness groups show smaller or negative divergences. Divergences between scores suggest positive spatial autocorrelation across European countries (Moran's I = 0.347, p = 0.036), indicating that stakeholders from neighboring countries tend toward more similar text-based stances towards AI safety concerns. Despite divergence, survey-reported concerns remain strongly associated with support for explainability across all divergence levels. These results demonstrate that LLM annotation reproducibility can coexist with poor construct correspondence and motivate validation procedures that distinguish reproducibility, construct validity, and communication context variation when LLMs are used as measurement instruments.
Abstract:High annotation reproducibility does not necessarily imply that an LLM-inferred measure captures the construct it is intended to measure. We test this distinction using a dataset from the European Commission's AI Act consultation, linking structured survey responses to free-text consultation submissions from the same stakeholders. LLM annotations of consultation submissions are highly reproducible (intraclass correlations > 0.99), yet show limited convergence with survey-reported measures of the nominal construct they were intended to approximate. Divergence between survey-and LLM-inferred text-based measures varies systematically across stakeholder groups: business associations express greater concern about AI risks in text-based consultations than in survey responses ({g} = +1.0), whereas public authorities and several nonbusiness groups show smaller or negative divergences. Divergences between scores suggest positive spatial autocorrelation across European countries (Moran's I = 0.347, p = 0.036), indicating that stakeholders from neighboring countries tend toward more similar text-based stances towards AI safety concerns. Despite divergence, survey-reported concerns remain strongly associated with support for explainability across all divergence levels. These results demonstrate that LLM annotation reproducibility can coexist with poor construct correspondence and motivate validation procedures that distinguish reproducibility, construct validity, and communication context variation when LLMs are used as measurement instruments.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY); Risk Management (q-fin.RM)
arXiv:2609.19866 [cs.AI]
(or arXiv:2609.19866v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.19866 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 300 words
Related coverage
Tech to Replace Animal Testing Is Almost Ready. Scientists Are Not
IEEE Spectrum · Sep 29, 2026
Mining Frames: Association Rule Mining as an Unsupervised Framework for News Framing Detection
arXiv · Oct 26, 2026
The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing
arXiv · Sep 9, 2026
Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6% WER (13.8% cWER)
arXiv · Jun 9, 2026