aiqbench analysis
This research introduces JusticeAxis, a novel benchmark and agent framework designed to balance rigid legal rule application with contextual discretion using multi-modal criminal case data. By employing a modular architecture where fact-finding and law-application agents operate under Bayesian constraints, the study demonstrates how frozen open-weight backbones can be effectively adapted to commercial standards without drifting into unsupported reasoning. The work addresses a critical gap in AI safety and alignment by providing a rigorous method to verify that automated judgments remain grounded in both statutory requirements and specific case circumstances.
JusticeAxis: Benchmarking Legal Judgment between Rigid Rule Application and Ungrounded Discretion
arXiv:2610.00353v1 Announce Type: new Abstract: A sound judgment applies the law to established facts and weighs the circumstances in which they arose. However, existing methods swing between rigid statute matching and ungrounded discretion, benchmarks score a label or a rubric, and the experience that would supply the balance stays unverified. We formalize legal judgment as a reference-anchored task, whose object is a single decision that stays tied to the statute and to the circumstances at once. We introduce JusticeAxis, 256 real-world criminal cases from 18 countries with audio, image, and text evidence, and three lawyer-written judgments for every case: the recorded one and one for each failure. We further propose JusticeAgent, a harness whose element agents establish the facts and whose judge agent applies the law under skills carrying experience of the circumstances. Skills are distilled from execution trajectories and admitted only under Bayesian credible bounds. Experiments show that failure turns direction with scale: open-weight backbones drift to unsupported grounds, frontier models to the statutory default. We further verify that JusticeAgent, as a simple yet effective plugin, carries a frozen open-weight backbone to commercial level. Project resources are available at https://github.com/beita6969/JusticeAxis.
Abstract:A sound judgment applies the law to established facts and weighs the circumstances in which they arose. However, existing methods swing between rigid statute matching and ungrounded discretion, benchmarks score a label or a rubric, and the experience that would supply the balance stays unverified. We formalize legal judgment as a reference-anchored task, whose object is a single decision that stays tied to the statute and to the circumstances at once. We introduce JusticeAxis, 256 real-world criminal cases from 18 countries with audio, image, and text evidence, and three lawyer-written judgments for every case: the recorded one and one for each failure. We further propose JusticeAgent, a harness whose element agents establish the facts and whose judge agent applies the law under skills carrying experience of the circumstances. Skills are distilled from execution trajectories and admitted only under Bayesian credible bounds. Experiments show that failure turns direction with scale: open-weight backbones drift to unsupported grounds, frontier models to the statutory default. We further verify that JusticeAgent, as a simple yet effective plugin, carries a frozen open-weight backbone to commercial level. Project resources are available at this https URL.
Artificial Intelligence (cs.AI)
arXiv:2610.00353 [cs.AI]
(or arXiv:2610.00353v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.00353 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 282 words
Related coverage
LiveLaw Supreme Court Half-Yearly Digest: January - June 2026 [Part 2]
livelaw.in · Sep 13, 2026
Introducing Mistral OCR 4
Mistral AI · Jun 23, 2026
Enterprise environments and training AI agents for real-world workflows
Snorkel AI Blog · Aug 3, 2026
JudgmentBench: Comparing Rubric and Preference Evaluation for Quality Assessment
Snorkel AI Blog · Jun 18, 2026