aiqbench analysis
This research establishes a theoretical trade-off in autoregressive forecasting, demonstrating that while long-horizon training yields better generalization to short-term predictions, it creates a rougher loss landscape that complicates optimization. The findings offer a principled framework for hyperparameter selection, balancing performance learnability against the geometric challenges of training dynamical systems.
Temporal horizons in forecasting: a performance-learnability trade-off
arXiv:2506.03889v3 Announce Type: replace Abstract: When training autoregressive models to forecast dynamical systems, a critical question arises: how far into the future should the model be trained to predict for optimal performance? In this work, we address this question by analyzing the relationship between the geometry of the loss landscape and the training time horizon. Using dynamical systems theory, we prove that loss minima for long horizons generalize well to short-term forecasts, whereas minima found on short horizons result in worse long-term predictions. However, we also prove that the loss landscape becomes rougher as the training horizon grows, making long-horizon training inherently challenging. We validate our theory through numerical experiments and discuss practical implications for selecting training horizons. Our results provide a principled foundation for hyperparameter optimization in autoregressive forecasting models.
Abstract:When training autoregressive models to forecast dynamical systems, a critical question arises: how far into the future should the model be trained to predict for optimal performance? In this work, we address this question by analyzing the relationship between the geometry of the loss landscape and the training time horizon. Using dynamical systems theory, we prove that loss minima for long horizons generalize well to short-term forecasts, whereas minima found on short horizons result in worse long-term predictions. However, we also prove that the loss landscape becomes rougher as the training horizon grows, making long-horizon training inherently challenging. We validate our theory through numerical experiments and discuss practical implications for selecting training horizons. Our results provide a principled foundation for hyperparameter optimization in autoregressive forecasting models.
Machine Learning (cs.LG); Chaotic Dynamics (nlin.CD)
arXiv:2506.03889 [cs.LG]
(or arXiv:2506.03889v3 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2506.03889 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 221 words
Related coverage
Echo Flow Networks
arXiv · Aug 6, 2026
Disentangling Heterogeneous Traffic Dynamics for Multi-Step Traffic Forecasting via Adaptive Spectral Decomposition
arXiv · Sep 23, 2026
CTIA-ODE: Continuous-Time Inter-Variable Attention for Long-Term Time Series Forecasting
arXiv · Oct 15, 2026
PhyMamba: Physics-Modulated Mamba for Robust Battery Health Prognostics
arXiv · Aug 31, 2026