aiqbench analysis
This research critically evaluates methods for detecting benchmark contamination in large language models by analyzing internal activation probes rather than relying on unavailable training data or test statistics. The study reveals that standard correction techniques introduce excessive statistical variance, causing false positives to track the size of control sets and rendering many previously significant findings non-significant when measurement uncertainty is properly propagated. Ultimately, the authors conclude that no definitive evidence of contamination was found in their audits once rigorous error bounds were applied, highlighting a gap between idealized simulations and real-world transformer behavior where surface decodability proxies degrade signal quality.
Excess Separability: Nuisance-Controlled Residual-Stream Probing for Benchmark Contamination Detection
arXiv:2608.12652v2 Announce Type: replace-cross Abstract: Benchmark contamination is diagnosed with n-gram overlap, likelihood-based membership inference, or canary strings, and each needs something usually unavailable: the training corpus, a well-chosen test statistic, or foresight at release. A recent alternative reads it off a linear probe on internal activations. We show the natural way to do this does not work, specify one that survives measurement, then find that the correction making it work carries more variance than the null it is tested against. The protocol reports a zero-sum contrast on the depth profile of probe accuracy, recentred on a level-matched placebo baseline, tested against a label-permutation null, with the reference set twice the size of the suspect set. Each choice replaces a simpler alternative we rejected on measurement. Reporting the level of excess separability rather than its shape makes the false positive rate track the size of the analyst's own control set, 0.03 to 0.99 under a true null. Contrasting against a flat depth profile rejects a true null 0.72 of the time when surface decodability rises with depth, and loses all power when it falls. On real transformers the protocol fails a test the simulations did not pose. The recentring subtracts an estimate, and the permutation null holds it fixed. Re-estimated across split seeds on four audits of contaminated checkpoints, its standard deviation is 1.30 to 1.56 times the null's own in every arm: what is subtracted to remove a bias is more variable than what it corrects. The one nominally significant result, p = 0.0075, becomes 0.0745 once that variance is propagated, and no verdict is issued. The simulations missed this because their surface key is the covariate driving item variation; on real text it is a proxy, and degrading key quality in simulation reproduces it. We add a companion measurement and a widened null. No arm shows contamination.
Abstract:Benchmark contamination is diagnosed with n-gram overlap, likelihood-based membership inference, or canary strings, and each needs something usually unavailable: the training corpus, a well-chosen test statistic, or foresight at release. A recent alternative reads it off a linear probe on internal activations. We show the natural way to do this does not work, specify one that survives measurement, then find that the correction making it work carries more variance than the null it is tested against. The protocol reports a zero-sum contrast on the depth profile of probe accuracy, recentred on a level-matched placebo baseline, tested against a label-permutation null, with the reference set twice the size of the suspect set. Each choice replaces a simpler alternative we rejected on measurement. Reporting the level of excess separability rather than its shape makes the false positive rate track the size of the analyst's own control set, 0.03 to 0.99 under a true null. Contrasting against a flat depth profile rejects a true null 0.72 of the time when surface decodability rises with depth, and loses all power when it falls. On real transformers the protocol fails a test the simulations did not pose. The recentring subtracts an estimate, and the permutation null holds it fixed. Re-estimated across split seeds on four audits of contaminated checkpoints, its standard deviation is 1.30 to 1.56 times the null's own in every arm: what is subtracted to remove a bias is more variable than what it corrects. The one nominally significant result, p = 0.0075, becomes 0.0745 once that variance is propagated, and no verdict is issued. The simulations missed this because their surface key is the covariate driving item variation; on real text it is a proxy, and degrading key quality in simulation reproduces it. We add a companion measurement and a widened null. No arm shows contamination.
23 pages, 11 figures, 8 tables. v2: measures the placebo baseline's own sampling variance, finds it exceeds the permutation null's in every audit, propagates it, and withdraws the one nominally significant result. Code and artefacts: this https URL
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2608.12652 [cs.CL]
(or arXiv:2608.12652v2 [cs.CL] for this version)
Full text under the publisher's open licence · 357 words
Related coverage
Links #5: 2026/07
LessWrong · Jul 31, 2026
Does Your LLM Trust You?
LessWrong · Aug 4, 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026
A Geometric Account of Activation Steering through Angle–Norm Decomposition
LessWrong · Jun 17, 2026