aiqbench analysis
This arXiv paper proposes DeCert, a training method for generative reward models (LLM-based judges). It supervises the reasoning behind a verdict, not just the verdict, by scoring whether rationales are valid and consistent with the verdict, without needing reference explanations. The reported gains are benchmark results across two model families, including up to 14.8 points on position-consistent accuracy, so readers should look for independent replication and how the method holds up outside the tested benchmarks.
Judging the Justification: Decision Certificate Supervision for Generative Reward Models
arXiv:2610.27528v1 Announce Type: new Abstract: Generative reward models (GenRMs) generate natural language rationales to justify preference judgments, yet a correct verdict does not guarantee a sound justification. Existing approaches often rely on reference explanations to supervise rationales, but obtaining such references can be costly and does not necessarily establish whether generated rationales support the resulting verdict. To address these limitations, we propose DeCert, a framework that evaluates the validity and verdict consistency of generated rationales without requiring reference explanations. Crucially, DeCert translates these assessments into preference supervision that accounts for both verdict correctness and justification quality. In particular, when multiple sampled judgments share the same correct verdict, differences in certificate quality enable DeCert to construct rationale preference pairs that distinguish stronger from weaker justifications. Extensive empirical studies across two model families and multiple in- and out-of-distribution benchmarks demonstrate that DeCert consistently outperforms strong generative judge baselines while improving rationale quality. Moreover, DeCert effectively mitigates positional bias, improving position-consistent accuracy by up to 14.8 points. These results highlight the importance of supervising not only which verdict is correct, but also how that verdict is justified.
Abstract:Generative reward models (GenRMs) generate natural language rationales to justify preference judgments, yet a correct verdict does not guarantee a sound justification. Existing approaches often rely on reference explanations to supervise rationales, but obtaining such references can be costly and does not necessarily establish whether generated rationales support the resulting verdict. To address these limitations, we propose DeCert, a framework that evaluates the validity and verdict consistency of generated rationales without requiring reference explanations. Crucially, DeCert translates these assessments into preference supervision that accounts for both verdict correctness and justification quality. In particular, when multiple sampled judgments share the same correct verdict, differences in certificate quality enable DeCert to construct rationale preference pairs that distinguish stronger from weaker justifications. Extensive empirical studies across two model families and multiple in- and out-of-distribution benchmarks demonstrate that DeCert consistently outperforms strong generative judge baselines while improving rationale quality. Moreover, DeCert effectively mitigates positional bias, improving position-consistent accuracy by up to 14.8 points. These results highlight the importance of supervising not only which verdict is correct, but also how that verdict is justified.
Computation and Language (cs.CL)
arXiv:2610.27528 [cs.CL]
(or arXiv:2610.27528v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.27528 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 274 words
Related coverage
Minimum-Cost Robust Evidence Certificates for Horn Reasoning under Correlated Source Failures
arXiv · Oct 20, 2026
No Certificate, No Categorical Speech Act: A Brouwerian Assertibility Constraint for Public Reason
arXiv · Jul 21, 2026
Lattice-Based Cryptography Explained
Quantum Zeitgeist · Aug 15, 2026
Europe’s Kurti Prize seeks young scientists using extreme conditions
Quantum Zeitgeist · Oct 13, 2026