analiza aiqbench
For time series foundation models, the choice of synthetic data generator is less critical than the strategy of mixing multiple generators, which consistently matches or outperforms the best single option. This approach effectively sidesteps the instability of generator rankings across different architectures, suggesting that corpus composition should be validated per model family rather than assumed to transfer. Practitioners should prioritize blending synthetic sources with real data to maximize pretraining efficacy without incurring the costs of extensive generator selection.
Mix, Don't Pick: Why Synthetic Corpus Composition Matters for Time Series Foundation Model Pretraining
arXiv:2606.09912v1 Announce Type: new Abstract: Choosing the wrong synthetic generator for time-series foundation model pretraining is costly: under identical training budgets, the best and worst generators produce up to a $2\times$ gap in forecasting error, yet the field has no principled way to make this choice. The problem is compounded by the fact that generator rankings are not stable across architectures: across 11 generator families evaluated on Chronos-T5-Mini and Moirai-Small trained from scratch, we find that which generators are useful depends on the model architecture. Rather than solving the generator selection problem, we sidestep it: a simple equal-weight mixture of all generators matches or beats the best individual generator for both architectures, and composing this mixture with real data yields the strongest pretraining corpora overall. Synthetic pretraining is therefore a corpus composition problem, not a generator selection problem, and composition choices should be validated per model family rather than assumed to transfer.
Abstract:Choosing the wrong synthetic generator for time-series foundation model pretraining is costly: under identical training budgets, the best and worst generators produce up to a $2\times$ gap in forecasting error, yet the field has no principled way to make this choice. The problem is compounded by the fact that generator rankings are not stable across architectures: across 11 generator families evaluated on Chronos-T5-Mini and Moirai-Small trained from scratch, we find that which generators are useful depends on the model architecture. Rather than solving the generator selection problem, we sidestep it: a simple equal-weight mixture of all generators matches or beats the best individual generator for both architectures, and composing this mixture with real data yields the strongest pretraining corpora overall. Synthetic pretraining is therefore a corpus composition problem, not a generator selection problem, and composition choices should be validated per model family rather than assumed to transfer.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.09912 [cs.LG]
(or arXiv:2606.09912v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.09912 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 242 cuvinte
Subiecte conexe
Multi-Hop Knowledge Composition is Bound by Pretraining Exposure
arXiv · 9 iun. 2026
Multi-Hop Knowledge Composition is Bound by Pretraining Exposure
arXiv · 17 sept. 2026
DISTAL: Distillation and Self-Supervised Pretraining for Structure-Agnostic Materials Property Prediction
arXiv · 2 sept. 2026
A Composition-Aware Pretraining Framework for Geospatial Foundation Models
arXiv · 1 sept. 2026