aiqbench analysis
LLM-based automated judges demonstrate significant inconsistency when evaluating safety across different criteria, languages, and domains, particularly struggling with nuanced issues in regulated sectors like finance. While they remain more reliable for detecting overt harms such as violence, practitioners must account for high inter-judge disagreement and linguistic style variations before deploying these evaluators in production environments. The study offers critical guidance for refining automated evaluation pipelines to ensure robustness in multi-dimensional safety assessments.
LLM Judges Inconsistently Disagree Across Safety Criteria and Harm Categories
arXiv:2605.31381v2 Announce Type: replace Abstract: We evaluate the consistency of automated judges in conducting a multi-dimensional safety evaluation in a reference-free setup. Our results indicate that Large Language Models are unreliable judges in identifying safety issues related to machine-generated advice in regulated domains such as finance, although they are more reliable at identifying more overt forms of unsafe/harmful content such as violence. The degree of inconsistency in a model's judgments can vary significantly by the chosen safety criteria and can be impacted by the language of the content and its linguistic style as well. Finally, there is high disagreement among different judges for the same output, across domains, safety criteria, and languages. These findings provide new insights on the practice of using LLMs as evaluators and offer several recommendations for practitioners on how to use automated judges in practical scenarios.
Abstract:We evaluate the consistency of automated judges in conducting a multi-dimensional safety evaluation in a reference-free setup. Our results indicate that Large Language Models are unreliable judges in identifying safety issues related to machine-generated advice in regulated domains such as finance, although they are more reliable at identifying more overt forms of unsafe/harmful content such as violence. The degree of inconsistency in a model's judgments can vary significantly by the chosen safety criteria and can be impacted by the language of the content and its linguistic style as well. Finally, there is high disagreement among different judges for the same output, across domains, safety criteria, and languages. These findings provide new insights on the practice of using LLMs as evaluators and offer several recommendations for practitioners on how to use automated judges in practical scenarios.
Computation and Language (cs.CL)
arXiv:2605.31381 [cs.CL]
(or arXiv:2605.31381v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2605.31381 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 228 words
Related coverage
Diagnosing Fine-Grained Inconsistency Classification in Financial Disclosure Text
arXiv · Jul 31, 2026
The Super Emotion Dataset
arXiv · Oct 21, 2026
Toward Automated Detection of Documentation Inconsistencies in Electronic Health Records
arXiv · Jul 28, 2026
GENEB: Why Genomic Models Are Hard to Compare
arXiv · Aug 6, 2026