Trend strength predicts when generative foundation models win: a power-controlled benchmark, a mechanism, and an actionable selection rule

Abstract:Pretrained generative foundation models cast forecasting as conditional generation from a learned predictive distribution and forecast unseen series zero-shot. We establish three results that turn their reported success into an actionable, mechanistic understanding. First (a positive benchmark result): on a power-controlled study of 1728 rolling-origin forecasts over 36 series from 19 datasets spanning the full range of STL trend strength (F_T in [0.17, 1.00]), a zero-shot Chronos model significantly outperforms four strong classical baselines -- drift, seasonal-naive, Theta, and additive Holt-Winters/ETS -- with the best mean MASE (1.187 vs. Theta 1.337, ETS 1.656; Friedman chi^2 = 46.08, p = 8.75e-09; Holm-corrected Wilcoxon p <= 0.015 against every baseline; a Nemenyi critical difference separating it from the classical pack). Second (a novel, quantified mechanism): a controlled synthetic experiment with a known trend-generating process shows why -- and reveals that the win does not come from better trend extrapolation. When the true trend is linear, damped, or exponential, additive ETS tracks the slope (slope-tracking ratio 1.02, 1.34, 0.98) whereas Chronos systematically under-extrapolates, behaving as a trend-shrinkage estimator (ratio 0.80, 0.49, 0.36). Third (an actionable selection rule): because the advantage is a shrinkage effect, it is predictable from trend strength alone -- the generative model wins 78% of low-trend series but only 44% of high-trend ones, and its edge over ETS is significant on the low-trend stratum (0.982 vs. 1.671, p = 0.002) yet a tie on the high-trend stratum (p = 0.18). Trend strength, computable before forecasting from the training context alone, is therefore a practical a-priori indicator of when to deploy a foundation model. We additionally document a calibration shortfall (80% intervals cover 0.77).

12 pages, 6 figures. Code and per-forecast data released

Applications (stat.AP); Machine Learning (cs.LG)

arXiv:2607.19383 [stat.AP]

(or arXiv:2607.19383v1 [stat.AP] for this version)

https://doi.org/10.48550/arXiv.2607.19383 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.