aiqbench analysis
This paper addresses the critical challenge of distinguishing genuine quantitative trading signals from statistical artifacts caused by extensive backtesting, a core problem in machine learning applications to finance. The authors propose an "injected-truth" protocol that validates whether admission gates actually filter noise or simply impose arbitrary thresholds, revealing that common criteria based on absolute returns often discard true signals while those using excess returns perform better but still face high false discovery rates in weak-signal regimes. For AI practitioners working in quantitative finance, this work provides essential insights into rigorous validation methods necessary to prevent overfitting and ensure model robustness.
On the Boundary of Admission Gates: An Injected-Truth Study of Falsification-First Selection in Quantitative Strategy Research
arXiv:2610.07701v1 Announce Type: new Abstract: Strategy research conflates two problems: finding a profitable rule, and establishing that the finding is not search luck. The latter calls for admission gates -- statistical criteria that must be satisfied before a conclusion is adopted -- yet whether gates work, and at what cost, remains untested. We introduce an injected-truth protocol with a random-admission control that adopts at the same rate as the gate; only if the gate beats this control does it carry information rather than merely raise a threshold. Across synthetic and real-calibrated panels, gates eliminate false discoveries in the weak-signal regime but cut adoption to 1--7%, and add nothing when signals are strong. Most importantly, criteria computed on absolute rather than excess returns silently reject every candidate, including true signals. Keywords: multiple testing, backtest overfitting, strategy admission, injected-truth validation, excess returns, false discovery rate
Abstract:Strategy research conflates two problems: finding a profitable rule, and establishing that the finding is not search luck. The latter calls for admission gates -- statistical criteria that must be satisfied before a conclusion is adopted -- yet whether gates work, and at what cost, remains untested. We introduce an injected-truth protocol with a random-admission control that adopts at the same rate as the gate; only if the gate beats this control does it carry information rather than merely raise a threshold. Across synthetic and real-calibrated panels, gates eliminate false discoveries in the weak-signal regime but cut adoption to 1--7%, and add nothing when signals are strong. Most importantly, criteria computed on absolute rather than excess returns silently reject every candidate, including true signals. Keywords: multiple testing, backtest overfitting, strategy admission, injected-truth validation, excess returns, false discovery rate
12 pages, 3 figures, 7 tables. Code and data to reproduce every result: this https URL
Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE)
arXiv:2610.07701 [cs.AI]
(or arXiv:2610.07701v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.07701 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 254 words
Related coverage
Article: Governing AI in the Cloud: A Practical Guide for Architects
InfoQ · Jun 15, 2026
Building trust in AI agents: A production-ready evaluation framework with Amazon Bedrock AgentCore
AWS · Oct 16, 2026
I don't think Claude is misaligned in 'Agentic Misalignment Summer 2026 - Motivated Mislabeling'
LessWrong · Jul 17, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026