aiqbench analysis
This research demonstrates that normalization strategies in causal, transformer-based time-series models critically impact both training convergence and forecasting accuracy. By addressing potential information leakage in efficient causal settings, the study highlights the necessity of selecting appropriate normalization techniques to maintain predictive performance on non-stationary data. Practitioners building large-scale forecasting systems must carefully evaluate these architectural choices to avoid subtle biases introduced during the training phase.
Does Normalization Choice Matter for Causal Large Time-Series Models?
arXiv:2606.09954v1 Announce Type: new Abstract: Large models for time-series forecasting have been emerged as a promising paradigm for training models on heterogeneous collections of signals. These models typically rely on causal autoregressive architectures, where each observation is sequentially predicted from past. In practice, real-world time-series exhibit non-stationarities, which significantly influence predictive performance. To mitigate this, normalization is commonly employed. However, in efficient causal settings it might induce information leakage from future observations during training. Recent alternatives, including causal normalization and statistics computed from initial observations, have been proposed to address this issue, but their practical implications remain insufficiently understood. In this work, we evaluate normalization strategies for transformer-based large time-series models trained with patching and efficient causal strategy. We showcase that normalization choice significantly influences both training convergence and forecasting performance.
Title:Does Normalization Choice Matter for Causal Large Time-Series Models?
Abstract:Large models for time-series forecasting have been emerged as a promising paradigm for training models on heterogeneous collections of signals. These models typically rely on causal autoregressive architectures, where each observation is sequentially predicted from past. In practice, real-world time-series exhibit non-stationarities, which significantly influence predictive performance. To mitigate this, normalization is commonly employed. However, in efficient causal settings it might induce information leakage from future observations during training. Recent alternatives, including causal normalization and statistics computed from initial observations, have been proposed to address this issue, but their practical implications remain insufficiently understood. In this work, we evaluate normalization strategies for transformer-based large time-series models trained with patching and efficient causal strategy. We showcase that normalization choice significantly influences both training convergence and forecasting performance.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.09954 [cs.LG]
(or arXiv:2606.09954v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.09954 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 232 words
