aiqbench analysis
The paper studies whether an LLM's wrong answers under certain instructions reflect a change in its internal judgment or a failure to express that judgment. It identifies a low-dimensional "expression subspace" without labels, then removes it from a linear probe, which raises AUROC substantially on negated entailment and safety questions. The results are promising but come from a single preprint with limited task coverage, so readers should watch for replication on other models and tasks before treating the method as a general fix for unreliable LLM outputs.
Isolating a Language Model's Judgment with a Label-Free Subspace
arXiv:2610.21256v1 Announce Type: cross Abstract: Language models are often asked to make judgments: whether a proposed answer is correct, whether one statement follows logically from another, or whether a response is safe. Yet changing the instructions that specify how the judgment should be expressed can make the final answer less reliable, even for the same input. We ask whether this reflects a change in internal judgment or a failure to express it. Comparing hidden-state vectors for the same input under different instructions, we find that their differences concentrate in a low-dimensional expression subspace. We identify this subspace without benchmark labels and remove its contribution from a linear predictor to obtain an isolated judgment. Across correctness, entailment, and safety, the isolated judgment remains reliable under instructions not used to identify the subspace, even when the model's final answer and the original predictor degrade. When the judgment question is negated, isolation raises the predictor's AUROC from 0.504 to 0.825 for entailment and from 0.480 to 0.891 for safety. We also find that judgment is readable in earlier layers, before the expression subspace begins to disrupt the predictor. Together, these results provide a way to separate judgment from expression and recover internal judgments even when the model's final answers are unreliable.
Abstract:Language models are often asked to make judgments: whether a proposed answer is correct, whether one statement follows logically from another, or whether a response is safe. Yet changing the instructions that specify how the judgment should be expressed can make the final answer less reliable, even for the same input. We ask whether this reflects a change in internal judgment or a failure to express it. Comparing hidden-state vectors for the same input under different instructions, we find that their differences concentrate in a low-dimensional expression subspace. We identify this subspace without benchmark labels and remove its contribution from a linear predictor to obtain an isolated judgment. Across correctness, entailment, and safety, the isolated judgment remains reliable under instructions not used to identify the subspace, even when the model's final answer and the original predictor degrade. When the judgment question is negated, isolation raises the predictor's AUROC from 0.504 to 0.825 for entailment and from 0.480 to 0.891 for safety. We also find that judgment is readable in earlier layers, before the expression subspace begins to disrupt the predictor. Together, these results provide a way to separate judgment from expression and recover internal judgments even when the model's final answers are unreliable.
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2610.21256 [cs.LG]
(or arXiv:2610.21256v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.21256 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 301 words
Related coverage
Detection engineering meets AI-driven adversary emulation
1Password · Oct 15, 2026
Reducing medical claims review time with AI on AWS: The EXL Medical IDP solution
AWS · Sep 21, 2026
Hands-free first notice of loss: Using Strands Agents and Amazon Bedrock AgentCore Browser Tool for intelligent claims intake
AWS · Jun 9, 2026
Agent Evaluation Metric for multi-turn conversations
AWS · Sep 10, 2026