aiqbench analysis
This research exposes a critical rigidity in LLM-as-judge frameworks, revealing that these evaluators often fail to update their safety assessments when provided with novel contextual information or shifting definitions. The study demonstrates that unless the new context aligns closely with the model's pre-existing internal priors, its judgment remains static regardless of external input. This finding is vital for developers relying on automated evaluation pipelines, as it highlights the risk of systematic bias in scaling AI safety protocols without addressing these inherent limitations.
Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of Evaluators
arXiv:2606.07874v2 Announce Type: replace Abstract: LLMs-as-judges are the primary way to evaluate safety at scale. Despite their importance, LLM-judges themselves are rarely evaluated beyond human agreement in simple, static benchmarks. We therefore investigate two under-explored but crucial properties of LLMs-as-judges: their sensitivity to in-context information, and their steerability to differing safety definitions, which may not align with their internal safety priors. We evaluate the safety judging abilities of 13 generalist LLMs and safety-specific judges, and investigate the impact of novel in-context information and changing safety definitions. We find that while LLM-judges can learn from new information, they are broadly unlikely to update their evaluations if the context or safety definition departs from their prior.
Abstract:LLMs-as-judges are the primary way to evaluate safety at scale. Despite their importance, LLM-judges themselves are rarely evaluated beyond human agreement in simple, static benchmarks. We therefore investigate two under-explored but crucial properties of LLMs-as-judges: their sensitivity to in-context information, and their steerability to differing safety definitions, which may not align with their internal safety priors. We evaluate the safety judging abilities of 13 generalist LLMs and safety-specific judges, and investigate the impact of novel in-context information and changing safety definitions. We find that while LLM-judges can learn from new information, they are broadly unlikely to update their evaluations if the context or safety definition departs from their prior.
Artificial Intelligence (cs.AI)
arXiv:2606.07874 [cs.AI]
(or arXiv:2606.07874v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.07874 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 201 words
Related coverage
Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of Evaluators
arXiv · Jun 9, 2026
Automating coherent long-form video generation
Google Research · Sep 24, 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · Oct 5, 2026
I don't think Claude is misaligned in 'Agentic Misalignment Summer 2026 - Motivated Mislabeling'
LessWrong · Jul 17, 2026