Evaluation Resolution Confounds Learning-Rule Comparisons in Model-Brain RSA of Early Visual Cortex

arXiv:2608.12408v2 Announce Type: replace-cross Abstract: Representational similarity analysis (RSA) is increasingly used to ask which learning rules give convolutional networks brain-like representations. Because biologically plausible rules such as feedback alignment, predictive coding and STDP do not scale, studies that include them train small networks on small images (typically 32x32 CIFAR) and then compare them to brain responses recorded for naturalistic stimuli modeled at far higher resolution. We find that a common result here -- that untrained or locally trained networks rival or beat backpropagation at early visual cortex -- depends strongly on the resolution at which the network is evaluated. The V1 gap between an untrained and a backprop-trained network widens from -0.001 +/- 0.007 at the 32 px training resolution to +0.044 +/- 0.006 at 224 px, growing monotonically across six resolutions (n = 5 seeds). It holds in human fMRI and, directionally, in single-seed macaque electrophysiology, along the training trajectory, and for an ImageNet ResNet-50 and a Swin-Tiny transformer trained at 224 px. We test four candidate mechanisms and none accounts for it: train/eval resolution matching, low-level Gabor and pixel structure, the normalization state of the untrained baseline, and convergence of the pooled descriptor toward a global brightness statistic. A fifth experiment locates it: capping image detail at the training resolution while letting the pooled positions grow 12-fold removes about 90% of the effect, so the dependence lives on the image-content axis. One control result is worth stating separately: a single scalar luminance value per image reaches rho = 0.074 against V1, matching the untrained network's 0.075, bounding what this comparison can resolve at V1 here. The one learning effect that holds across resolution sits at LOC. Comparisons at early visual cortex must control, and report, the evaluation resolution.

Abstract:Representational similarity analysis (RSA) is increasingly used to ask which learning rules give convolutional networks brain-like representations. Because biologically plausible rules such as feedback alignment, predictive coding and STDP do not scale, studies that include them train small networks on small images (typically 32x32 CIFAR) and then compare them to brain responses recorded for naturalistic stimuli modeled at far higher resolution. We find that a common result here -- that untrained or locally trained networks rival or beat backpropagation at early visual cortex -- depends strongly on the resolution at which the network is evaluated. The V1 gap between an untrained and a backprop-trained network widens from -0.001 +/- 0.007 at the 32 px training resolution to +0.044 +/- 0.006 at 224 px, growing monotonically across six resolutions (n = 5 seeds). It holds in human fMRI and, directionally, in single-seed macaque electrophysiology, along the training trajectory, and for an ImageNet ResNet-50 and a Swin-Tiny transformer trained at 224 px. We test four candidate mechanisms and none accounts for it: train/eval resolution matching, low-level Gabor and pixel structure, the normalization state of the untrained baseline, and convergence of the pooled descriptor toward a global brightness statistic. A fifth experiment locates it: capping image detail at the training resolution while letting the pooled positions grow 12-fold removes about 90% of the effect, so the dependence lives on the image-content axis. One control result is worth stating separately: a single scalar luminance value per image reaches rho = 0.074 against V1, matching the untrained network's 0.075, bounding what this comparison can resolve at V1 here. The one learning effect that holds across resolution sits at LOC. Comparisons at early visual cortex must control, and report, the evaluation resolution.

13 pages, 7 figures. v2: text corrections only; no figure and no result changes. Corrects the fMRI RDM description (single-trial, not trial-averaged), the reproducibility bound in Sec. 4.1, the nondeterminism criterion, the layer-to-ROI mapping, and twelve printed values in the last digit. Withdraws the noise-ceiling estimate and the "69% of ceiling" figure. See note after abstract

Neurons and Cognition (q-bio.NC); Machine Learning (cs.LG)

arXiv:2608.12408 [q-bio.NC]

(or arXiv:2608.12408v2 [q-bio.NC] for this version)

aiqbench analysis

Evaluation Resolution Confounds Learning-Rule Comparisons in Model-Brain RSA of Early Visual Cortex

Related coverage