analiza aiqbench
This study critically evaluates the utility of frozen forecasting foundation models as feature extractors for time series classification, revealing that their pretrained representations often fail to outperform strong non-pretrained baselines. The research underscores that gains from pretraining are marginal when compared to robust task-specific architectures, suggesting limited added value for classification tasks without fine-tuning or specific feature restoration. These findings provide essential benchmarks for researchers assessing the true transferability of forecasting models in low-data regimes.
Big Models, Small Gains: Frozen Forecasters as Classifiers on MONSTER
arXiv:2611.06988v1 Announce Type: new Abstract: Forecasting foundation models are increasingly used as frozen feature extractors for time series classification, but it remains unclear whether their pretrained representations offer an advantage over strong classifiers that do not require pretraining. This raises an important question: do these representations provide useful information beyond what the model architecture alone can capture, particularly when labelled data are scarce? We present a controlled benchmarking of two forecasting models, Reverso and Chronos-Bolt, on six large datasets from the MONSTER benchmark, using 1%, 10%, and 100% of the available training labels. We compare their frozen representations against four established time series classification methods and random-weight versions of the same forecasting models. Our results show that pretraining improves classification accuracy over random-weight counterparts on five of the six datasets, with gains of up to 23 percentage points. However, the frozen representations do not reliably outperform the strongest non-pretrained baseline in any setting: they perform worse in 15 of 17 settings and tie in the remaining two. Further experiments show that restoring amplitude information recovers part of the performance gap, while fine-tuning and forecast-error features do not close it. These findings highlight the importance of strong task specific baselines and random weight controls when evaluating forecasting foundation models for classification. Our code is available at https://github.com/Rachet2006/frozen-forecasters-monster.
Abstract:Forecasting foundation models are increasingly used as frozen feature extractors for time series classification, but it remains unclear whether their pretrained representations offer an advantage over strong classifiers that do not require pretraining. This raises an important question: do these representations provide useful information beyond what the model architecture alone can capture, particularly when labelled data are scarce? We present a controlled benchmarking of two forecasting models, Reverso and Chronos-Bolt, on six large datasets from the MONSTER benchmark, using 1%, 10%, and 100% of the available training labels. We compare their frozen representations against four established time series classification methods and random-weight versions of the same forecasting models. Our results show that pretraining improves classification accuracy over random-weight counterparts on five of the six datasets, with gains of up to 23 percentage points. However, the frozen representations do not reliably outperform the strongest non-pretrained baseline in any setting: they perform worse in 15 of 17 settings and tie in the remaining two. Further experiments show that restoring amplitude information recovers part of the performance gap, while fine-tuning and forecast-error features do not close it. These findings highlight the importance of strong task specific baselines and random weight controls when evaluating forecasting foundation models for classification. Our code is available at this https URL.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2611.06988 [cs.LG]
(or arXiv:2611.06988v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2611.06988 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 308 cuvinte
Subiecte conexe
Alignment Forecasting: Predicting Misalignment from Training Data
LessWrong · 25 sept. 2026
Foreclassing: From Forecasts to Decisions
arXiv · 28 oct. 2026
Reinforcement Learning for LLM-based Event Forecasting
arXiv · 27 oct. 2026
Reinforcement Learning for LLM-based Event Forecasting
arXiv · 16 iun. 2026