aiqbench analysis
This paper introduces MixBench-TS, a new benchmark addressing the critical gap in multivariate time series forecasting where standard datasets lack sufficient cross-channel coupling to properly evaluate channel-mixing models. The authors demonstrate that existing evaluation sets are misleadingly easy for independent models, whereas their proposed benchmark forces models to learn genuine non-linear dependencies between channels. This work is essential for researchers developing and testing advanced architectures designed to capture complex inter-variable dynamics in time series data.
MixBench-TS: A Multivariate Time Series Forecasting Benchmark Where Channel Mixing Pays Off
arXiv:2609.32656v1 Announce Type: cross Abstract: Multivariate Time Series Forecasting (MTSF) models that mix information across channels assume that the past of one channel carries information about the future of another. Yet they are evaluated on a small fixed set of standard datasets whose cross-channel structure is rarely examined. We ask two questions: "How can we reliably measure lagged, non-linear, and joint coupling in MTSF datasets?" and "Do the standard datasets actually have such coupling?" To answer the first, we test four candidate measures on synthetic datasets with planted ground-truth coupling: Granger Causality (GC), Transfer Entropy (TE), lagged Mutual Information (MI), and the CD gain, a model-based measure we introduce that compares a channel-dependent (CD) model to its channel-independent (CI) variant. Only lagged MI and the CD gain recover every planted coupling. For the second question, the answer is a definite no, as the standard datasets have a median of only 23% lagged-coupled channel pairs and a median CD gain of -4.9%, compared to 78% and +4.6% on chaotic ODE systems. We therefore propose MixBench-TS, a benchmark of 10 real-world datasets with a median of 55.5% lagged-coupled pairs and a median CD gain of +1.7%. Across six state-of-the-art models tuned under one protocol, CI models win on 10/10 (MSE) and 8/10 (MAE) standard datasets, but on only 3/10 and 2/10 MixBench-TS datasets. We recommend using our benchmark for evaluating new CD models. Moreover, we propose profiling new datasets with lagged MI and the CD gain before using them to evaluate multivariate models. Code and data are available at https://anonymous.4open.science/r/mixbench-ts-B027.
Abstract:Multivariate Time Series Forecasting (MTSF) models that mix information across channels assume that the past of one channel carries information about the future of another. Yet they are evaluated on a small fixed set of standard datasets whose cross-channel structure is rarely examined. We ask two questions: "How can we reliably measure lagged, non-linear, and joint coupling in MTSF datasets?" and "Do the standard datasets actually have such coupling?" To answer the first, we test four candidate measures on synthetic datasets with planted ground-truth coupling: Granger Causality (GC), Transfer Entropy (TE), lagged Mutual Information (MI), and the CD gain, a model-based measure we introduce that compares a channel-dependent (CD) model to its channel-independent (CI) variant. Only lagged MI and the CD gain recover every planted coupling. For the second question, the answer is a definite no, as the standard datasets have a median of only 23% lagged-coupled channel pairs and a median CD gain of -4.9%, compared to 78% and +4.6% on chaotic ODE systems. We therefore propose MixBench-TS, a benchmark of 10 real-world datasets with a median of 55.5% lagged-coupled pairs and a median CD gain of +1.7%. Across six state-of-the-art models tuned under one protocol, CI models win on 10/10 (MSE) and 8/10 (MAE) standard datasets, but on only 3/10 and 2/10 MixBench-TS datasets. We recommend using our benchmark for evaluating new CD models. Moreover, we propose profiling new datasets with lagged MI and the CD gain before using them to evaluate multivariate models. Code and data are available at this https URL.
Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2609.32656 [cs.AI]
(or arXiv:2609.32656v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.32656 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 352 words
Related coverage
VFEM: Visual Feature Empowered Multivariate Time Series Forecasting with Cross-Modal Fusion
arXiv · Jun 9, 2026
Structured Latent Space Modeling over Multi-Scale Temporal Patches for Multivariate Time Series Forecasting
arXiv · Jul 23, 2026
A Hybrid Attention Model Learning Unified Time-aware Patch Representation for Irregular Multivariate Time Series Forecasting
arXiv · Sep 22, 2026
Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitives
arXiv · Oct 10, 2026