aiqbench analysis
This research exposes a critical flaw in how we evaluate Large Language Models, revealing that standard accuracy metrics often mask the absence of genuine reasoning regarding hidden constraints. By introducing paired benchmarks that compare decisions with and without specific conditions, the study demonstrates that models frequently rely on heuristic caution rather than true logical deduction. Consequently, current evaluation methods may be misleadingly optimistic about a model's ability to handle complex, context-dependent scenarios.
The Model Says Walk: Measuring whether LLMs Condition on Hidden Constraints
arXiv:2603.29025v4 Announce Type: replace Abstract: Asked whether to walk or drive to a car wash 50 m away, most language models say walk, forgetting that the car has to be there. Such failures are usually measured by accuracy on questions where the hidden constraint applies. We show that this is misleading: a model can answer these questions correctly without reasoning about the constraint at all, simply by favoring the cautious option. We instead ask whether a model's decision changes when the constraint is removed. We test this at three levels of control: log-probability sweeps on open-weight models, a 500-prompt stress test, and CORE, a new human-validated benchmark of minimal constraint-present/constraint-absent pairs. The picture is consistent. The constraint nudges decisions rather than governing them. Models still miss presence constraints like the car wash, yet elsewhere they apply constraints that are not there. As a result, standard accuracy flatters all ten models we evaluate and reorders their ranking, and prompting fixes that look effective largely vanish under paired scoring. Claims about hidden-constraint reasoning need paired evidence.
Abstract:Asked whether to walk or drive to a car wash 50 m away, most language models say walk, forgetting that the car has to be there. Such failures are usually measured by accuracy on questions where the hidden constraint applies. We show that this is misleading: a model can answer these questions correctly without reasoning about the constraint at all, simply by favoring the cautious option. We instead ask whether a model's decision changes when the constraint is removed. We test this at three levels of control: log-probability sweeps on open-weight models, a 500-prompt stress test, and CORE, a new human-validated benchmark of minimal constraint-present/constraint-absent pairs. The picture is consistent. The constraint nudges decisions rather than governing them. Models still miss presence constraints like the car wash, yet elsewhere they apply constraints that are not there. As a result, standard accuracy flatters all ten models we evaluate and reorders their ranking, and prompting fixes that look effective largely vanish under paired scoring. Claims about hidden-constraint reasoning need paired evidence.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2603.29025 [cs.CL]
(or arXiv:2603.29025v4 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2603.29025 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 265 words
Related coverage
Game Design Creativity evals
LessWrong · Oct 12, 2026
Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk
Latent Space · Oct 8, 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · Oct 5, 2026
Links #3: 2026/06 Part 1
LessWrong · Jun 15, 2026