analiza aiqbench
CITRAS addresses a key limitation in deep learning time series forecasting by resolving length discrepancies between observed and known future covariates. The model introduces two specific mechanisms, KV Shift and Attention Score Smoothing, to flexibly integrate these variables while maintaining strong autoregressive capabilities. Experimental results across diverse real-world datasets demonstrate its effectiveness in leveraging both cross-variate and cross-time dependencies for improved accuracy.
CITRAS: Covariate-Informed Transformer for Time Series Forecasting
arXiv:2503.24007v4 Announce Type: replace Abstract: In time series forecasting, covariates represent external factors that influence target variables. Some covariates are observable only in the past (observed covariates, such as recorded weather data), while others are known in advance (known covariates, such as calendar events or discount schedules). Although covariates have the potential to enhance forecasting performance, most deep learning-based forecasting models struggle to address the length discrepancy between variables caused by the future portion of known covariates and fail to leverage them flexibly. Moreover, capturing dependencies between target variables and covariates is non-trivial, as models must accurately reflect the local impact of covariates while simultaneously modeling global cross-variate dependencies. To address these challenges, we propose CITRAS, a decoder-only Transformer that flexibly integrates multiple target variables, observed covariates, and known covariates. While preserving strong autoregressive modeling capabilities, CITRAS introduces two novel mechanisms within patch-wise cross-variate attention: Key-Value (KV) Shift and Attention Score Smoothing. KV Shift seamlessly incorporates the future portion of known covariates into the forecasting process by aligning them with target variables based on their concurrent dependencies. Attention Score Smoothing refines locally accurate patch-wise cross-variate dependencies into global variate-level dependencies by smoothing the historical attention scores. Experimentally, CITRAS demonstrates strong performance across a wide range of real-world datasets in both covariate-informed and multivariate settings, showcasing its versatile ability to leverage cross-variate and cross-time dependencies for improved forecasting accuracy.
Abstract:In time series forecasting, covariates represent external factors that influence target variables. Some covariates are observable only in the past (observed covariates, such as recorded weather data), while others are known in advance (known covariates, such as calendar events or discount schedules). Although covariates have the potential to enhance forecasting performance, most deep learning-based forecasting models struggle to address the length discrepancy between variables caused by the future portion of known covariates and fail to leverage them flexibly. Moreover, capturing dependencies between target variables and covariates is non-trivial, as models must accurately reflect the local impact of covariates while simultaneously modeling global cross-variate dependencies. To address these challenges, we propose CITRAS, a decoder-only Transformer that flexibly integrates multiple target variables, observed covariates, and known covariates. While preserving strong autoregressive modeling capabilities, CITRAS introduces two novel mechanisms within patch-wise cross-variate attention: Key-Value (KV) Shift and Attention Score Smoothing. KV Shift seamlessly incorporates the future portion of known covariates into the forecasting process by aligning them with target variables based on their concurrent dependencies. Attention Score Smoothing refines locally accurate patch-wise cross-variate dependencies into global variate-level dependencies by smoothing the historical attention scores. Experimentally, CITRAS demonstrates strong performance across a wide range of real-world datasets in both covariate-informed and multivariate settings, showcasing its versatile ability to leverage cross-variate and cross-time dependencies for improved forecasting accuracy.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2503.24007 [cs.LG]
(or arXiv:2503.24007v4 [cs.LG] for this version)
IEEE Access, vol. 14, pp. 77983-77998, 2026
https://doi.org/10.1109/ACCESS.2026.3695717 DOI(s) linking to related resources
Text integral sub licența deschisă a publicației · 250 cuvinte
Subiecte conexe
CITRAS-FM: Tiny Time Series Foundation Model for Covariate-Informed Zero-Shot Forecasting
arXiv · 10 iun. 2026
DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters
arXiv · 8 oct. 2026
A Lightweight Plug-in Gate for Transformer-Based Time-Series Forecasters
arXiv · 21 sept. 2026
Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting
arXiv · 27 iul. 2026