analiza aiqbench
This research identifies a critical failure mode in safety-critical AI systems where narrow task conditioning suppresses the reporting of unanticipated hazards, effectively creating an "Inattentional Gap" analogous to human blindness. The study demonstrates that this suppression persists across various model scales and architectures, suggesting it is a fundamental behavioral trait rather than a scaling issue, though one flagship model exhibited a robust override capability. For deployment in high-stakes environments like radiology, the authors propose adopting reporting-complete evaluation protocols as a mandatory admission criterion to ensure systems do not ignore co-present dangers simply because they were outside their specific instruction set.
The inattentional gap in task conditioned AI models that omit otherwise reportable safety critical signals
arXiv:2606.26529v3 Announce Type: replace-cross Abstract: AI in radiology and other safety-critical workflows is evaluated on the hazards it is told to find, yet harm arises disproportionately from hazards no one specified. We show that conditioning a language or vision model on a narrow task suppresses its reporting of co-present, safety-critical signals it can otherwise report, a behavioral analogue of human inattentional blindness. Across radiology text scenarios and thoracic-image vision tasks, ordinary focused instructions suppressed reporting by up to 0.92; the gap ranged from minimal to complete across seven models, did not vary monotonically with scale, and persisted in a reasoning model, while one flagship model showed a robust safety-reporting override. We term this dissociation the Inattentional Gap: a system can score near-perfectly on specified hazards while omitting co-present safety-critical hazards. In a 24-scenario probe, an independent open-ended critic restored every omitted finding. We propose reporting-complete evaluation as an admission criterion for safety-critical deployment.
arXiv:2606.26529 (cs)
Abstract:AI in radiology and other safety-critical workflows is evaluated on the hazards it is told to find, yet harm arises disproportionately from hazards no one specified. We show that conditioning a language or vision model on a narrow task suppresses its reporting of co-present, safety-critical signals it can otherwise report, a behavioral analogue of human inattentional blindness. Across radiology text scenarios and thoracic-image vision tasks, ordinary focused instructions suppressed reporting by up to 0.92; the gap ranged from minimal to complete across seven models, did not vary monotonically with scale, and persisted in a reasoning model, while one flagship model showed a robust safety-reporting override. We term this dissociation the Inattentional Gap: a system can score near-perfectly on specified hazards while omitting co-present safety-critical hazards. In a 24-scenario probe, an independent open-ended critic restored every omitted finding. We propose reporting-complete evaluation as an admission criterion for safety-critical deployment.
62 pages (31-page article and 31-page supplementary information), 8 figures, 4 tables. v3: corrects the author metadata to the sole author, Kwan Soo Shin; revised title and abstract; adds cross-vendor and flagship validation, signal-detection and specified-task controls, and dual-process probes. Reproducibility deposit: doi:https://doi.org/10.5281/zenodo.20826823
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2606.26529 [cs.CL]
(or arXiv:2606.26529v3 [cs.CL] for this version)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 286 cuvinte
Subiecte conexe
Always ask your agent to be honest
LessWrong · 16 sept. 2026
[AINews] not much happened today
Latent Space · 10 sept. 2026
Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals
METR · 27 sept. 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · 5 oct. 2026