aiqbench analysis
The paper argues that generative safety guards can be judged correct internally yet still fail in deployment, because outputs are unparseable and fallback logic then allows or blocks requests incorrectly. Its fail-open replay shows the gap is large: a parser mismatch let 95.5% of harmful requests through despite correct guard verdicts, which drops to 0% once the parser matches the output format. For anyone building or evaluating LLM-based moderation, the takeaway is that guard accuracy alone does not determine safety, and the parsing and fallback pipeline needs to be tested as part of the system.
REA: From Correct LLM Judgments to Safe Decision Delivery
arXiv:2610.17473v1 Announce Type: new Abstract: Generative safety guards must classify requests correctly and return outputs that applications can parse. Reasoning guards can improve classification accuracy while producing outputs that applications cannot parse. When applications cannot extract a correct prediction, fallback handling can allow harmful requests or block safe ones. We introduce Resolve--Express--Act (REA), a framework that traces judgments through parsing and fallback to application actions. Experiments on classifiers and safety guards show that correct predictions can appear in incorrectly formatted outputs. Parsing the safety verdict preserves the reasoning guard's accuracy advantage, while stricter explanation requirements reject additional correct predictions. Accepting formatting variations recovers most rejected correct predictions. Held-out tests confirm that output requirements and generation limits affect whether correct predictions can be parsed. In our fail-open replay, matching the parser to the output and fixing formatting reduced the fraction of harmful requests allowed despite correct guard judgments from 95.5% to 0%. A fixed-output comparison shows that model-based extraction can preserve guard verdicts, but is not automatically reliable; deterministic normalization already repairs the observed native-format failures. Fallback replay shows that the same correct prediction can lead to different safety actions depending on the parser and fallback policy, including allowing requests the guard correctly identifies as harmful.
Abstract:Generative safety guards must classify requests correctly and return outputs that applications can parse. Reasoning guards can improve classification accuracy while producing outputs that applications cannot parse. When applications cannot extract a correct prediction, fallback handling can allow harmful requests or block safe ones. We introduce Resolve--Express--Act (REA), a framework that traces judgments through parsing and fallback to application actions. Experiments on classifiers and safety guards show that correct predictions can appear in incorrectly formatted outputs. Parsing the safety verdict preserves the reasoning guard's accuracy advantage, while stricter explanation requirements reject additional correct predictions. Accepting formatting variations recovers most rejected correct predictions. Held-out tests confirm that output requirements and generation limits affect whether correct predictions can be parsed. In our fail-open replay, matching the parser to the output and fixing formatting reduced the fraction of harmful requests allowed despite correct guard judgments from 95.5% to 0%. A fixed-output comparison shows that model-based extraction can preserve guard verdicts, but is not automatically reliable; deterministic normalization already repairs the observed native-format failures. Fallback replay shows that the same correct prediction can lead to different safety actions depending on the parser and fallback policy, including allowing requests the guard correctly identifies as harmful.
Artificial Intelligence (cs.AI)
arXiv:2610.17473 [cs.AI]
(or arXiv:2610.17473v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.17473 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 294 words
Related coverage
Amazon Research Awards recipients announced
Amazon Science · May 27, 2026
Hint-based CoT faithfulness evals still mostly work on Claude
LessWrong · Jul 30, 2026
Built from the inside out: How AWS Professional Services became a frontier team first
AWS · Jun 12, 2026
Amazon is investing in the Lean Focused Research Organization
Amazon Science · Jul 26, 2026