aiqbench analysis
This research introduces a rigorous framework for decomposing standardized forecasts into target-aligned and uncorrelated components, offering critical insights for evaluating the true diversity of predictive models beyond simple error correlation. By distinguishing genuine improvement from mere dilution, the study provides a cautious selection rule that prevents equal-weight aggregation strategies from mistakenly rewarding non-informative signals. The methodology is further validated through applications involving language-model forecasts for equity rankings and mechanical signals for exchange-traded funds.
Target alignment, dilution and forecast selection when cross-sectional forecasts share a common target
arXiv:2609.26303v1 Announce Type: cross Abstract: Forecasters often score the same units per date against one standardized realized outcome. We show that every standardized forecast splits exactly into a component aligned with this common target and a component uncorrelated with it. Three consequences follow: forecast-error correlation largely mirrors forecast correlation and is therefore a poor measure of diversity; an equally weighted combination beats a no-information forecast only when average alignment is large relative to the combination's dispersion; and the gain from adding a forecaster separates into genuine improvement and mere dilution, which equal-weight admission can mistakenly reward. We develop a cautious selection rule, study it in simulations, and apply it to language-model forecasts of US equity rankings and mechanical signals ranking exchange-traded funds. Selection removes most dilution losses, but no combination beats the no-information forecast.
Abstract:Forecasters often score the same units per date against one standardized realized outcome. We show that every standardized forecast splits exactly into a component aligned with this common target and a component uncorrelated with it. Three consequences follow: forecast-error correlation largely mirrors forecast correlation and is therefore a poor measure of diversity; an equally weighted combination beats a no-information forecast only when average alignment is large relative to the combination's dispersion; and the gain from adding a forecaster separates into genuine improvement and mere dilution, which equal-weight admission can mistakenly reward. We develop a cautious selection rule, study it in simulations, and apply it to language-model forecasts of US equity rankings and mechanical signals ranking exchange-traded funds. Selection removes most dilution losses, but no combination beats the no-information forecast.
Econometrics (econ.EM); Machine Learning (cs.LG); Statistical Finance (q-fin.ST); Methodology (stat.ME)
arXiv:2609.26303 [econ.EM]
(or arXiv:2609.26303v1 [econ.EM] for this version)
https://doi.org/10.48550/arXiv.2609.26303 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 230 words
Related coverage

News
Quantum Computing Report · Jul 25, 2026

Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
AI Alignment Forum · Aug 3, 2026

Evaluating multi-agent systems for explainability and helpfulness with Amazon Bedrock AgentCore
AWS · Oct 5, 2026
The Continual Constitutional Convention: How language models revise the documents that govern them
AI Alignment Forum · Oct 13, 2026