aiqbench analysis
This research exposes a critical flaw in evaluating brain foundation models, demonstrating that high probe accuracy often stems from site-specific artifacts inherent in the raw MRI data rather than learned clinical knowledge. The study proves that pretraining does not generate these signals, as untrained models and even raw images can predict acquisition sites with similar precision. Consequently, researchers must adopt mandatory negative controls—testing on untrained models and raw inputs—to validate that their probes genuinely reflect model capabilities.
Site Is Decodable Before Pretraining: Negative Controls for Probing Frozen Brain-MRI Foundation Models
arXiv:2608.10295v2 Announce Type: replace-cross Abstract: Brain foundation models are often tested with a probe. The model is frozen, a simple classifier is trained on its output, and the classifier's accuracy is taken as a sign of what pretraining learned. We show that this reading can be wrong unless two controls are reported with it. We probed three frozen 3-D brain-MRI models at five depths, on two independent multi-site cohorts (ABIDE-I and ABIDE-II). The site where a scan was acquired could be predicted at about 0.9, on a scale where 0 is chance and 1 is perfect. No clinical or demographic target came close. An untrained copy of the same model predicted site just as well on both cohorts. Untrained models of three architectures, each built with three random seeds, did the same on ABIDE-I. The raw image, shrunk to 12x12x12 voxels and given to no model at all, predicted site at 0.95. Pretraining did not create the site signal. It was already in the scans. We also looked for any clinical target on which pretraining beats an untrained model, and found no reliable case. Adjusting for age, sex and diagnosis left site prediction almost unchanged, and so did a 4.6-fold increase in the number of voxels. Resting-state fMRI from the same people showed the same pattern. Site was the most predictable attribute of brain connectivity (0.81), and it survived five untrained layers while age did not. We recommend reporting two controls with every probe of a brain foundation model: running the probe again on the untrained model, and on the raw input. We release the code for both.
Abstract:Brain foundation models are often tested with a probe. The model is frozen, a simple classifier is trained on its output, and the classifier's accuracy is taken as a sign of what pretraining learned. We show that this reading can be wrong unless two controls are reported with it. We probed three frozen 3-D brain-MRI models at five depths, on two independent multi-site cohorts (ABIDE-I and ABIDE-II). The site where a scan was acquired could be predicted at about 0.9, on a scale where 0 is chance and 1 is perfect. No clinical or demographic target came close. An untrained copy of the same model predicted site just as well on both cohorts. Untrained models of three architectures, each built with three random seeds, did the same on ABIDE-I. The raw image, shrunk to 12x12x12 voxels and given to no model at all, predicted site at 0.95. Pretraining did not create the site signal. It was already in the scans. We also looked for any clinical target on which pretraining beats an untrained model, and found no reliable case. Adjusting for age, sex and diagnosis left site prediction almost unchanged, and so did a 4.6-fold increase in the number of voxels. Resting-state fMRI from the same people showed the same pattern. Site was the most predictable attribute of brain connectivity (0.81), and it survived five untrained layers while age did not. We recommend reporting two controls with every probe of a brain foundation model: running the probe again on the untrained model, and on the raw input. We release the code for both.
17 pages, 6 figures, 8 tables. v2: adds resting-state fMRI, reports each clinical target separately, corrects single-split results, rewritten for clarity. Code: this https URL
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
I.2.6; I.4.7; J.3
arXiv:2608.10295 [cs.CV]
(or arXiv:2608.10295v2 [cs.CV] for this version)
Full text under the publisher's open licence · 308 words
Related coverage
Stress-Testing EEG Foundation Models for Clinical Decoding: Dataset Identity and Targeted Negative Controls
arXiv · Jul 28, 2026
BrainG3N: A Dual-Purpose Tokenizer for Controllable 3D Brain MRI Generation
arXiv · Jul 31, 2026
Encoded but Not Actionable: Auditing the Decode-Generate-Steer Gap in Frozen LLMs for Geometric Constraints
arXiv · Aug 19, 2026
FLIP: Final Layer Inference-Time Probing for Vision-Language Models
arXiv · Sep 28, 2026