aiqbench analysis
The study reveals a critical limitation in current safety evaluation pipelines: while LLM-judges can absorb new information, they largely fail to adjust their assessments when contextual cues or safety definitions conflict with their internal priors. This rigidity suggests that relying solely on static benchmarks or standard in-context learning may not be sufficient for robust safety auditing at scale. Practitioners must account for these evaluator biases when designing automated safety protocols to avoid systematic blind spots.
Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of Evaluators
arXiv:2606.07874v1 Announce Type: new Abstract: LLMs-as-judges are the only way to evaluate safety at scale. Despite their importance, LLM-judges themselves are rarely evaluated beyond human agreement in simple, static benchmarks. We therefore investigate two under-explored but crucial properties of LLMs-as-judges: their susceptibility to relying on in context-information, and their steerability to differing safety definitions, which may not align with their internal safety priors. We evaluate the safety judging abilities of many generalist LLMs and safety-specific judges, and investigate the impact of task demonstrations, novel in-context information, and changing safety definitions. We find that while LLM-judges can learn from new information, they are broadly unlikely to adjust their evaluations if the context or safety definition contradicts their prior.
Abstract:LLMs-as-judges are the only way to evaluate safety at scale. Despite their importance, LLM-judges themselves are rarely evaluated beyond human agreement in simple, static benchmarks. We therefore investigate two under-explored but crucial properties of LLMs-as-judges: their susceptibility to relying on in context-information, and their steerability to differing safety definitions, which may not align with their internal safety priors. We evaluate the safety judging abilities of many generalist LLMs and safety-specific judges, and investigate the impact of task demonstrations, novel in-context information, and changing safety definitions. We find that while LLM-judges can learn from new information, they are broadly unlikely to adjust their evaluations if the context or safety definition contradicts their prior.
Artificial Intelligence (cs.AI)
arXiv:2606.07874 [cs.AI]
(or arXiv:2606.07874v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.07874 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 206 words
Related coverage
Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of Evaluators
arXiv · Oct 7, 2026
Automating coherent long-form video generation
Google Research · Sep 24, 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · Oct 5, 2026
I don't think Claude is misaligned in 'Agentic Misalignment Summer 2026 - Motivated Mislabeling'
LessWrong · Jul 17, 2026