aiqbench analysis
This study challenges the prevailing narrative that AI weather models inherently fail at predicting extreme events, finding instead that performance varies significantly by specific model architecture and variable rather than by the underlying methodology alone. The research highlights a universal bias across both AI and physical models to dampen extremes, suggesting that tail errors are systemic to current forecasting approaches rather than a unique flaw of machine learning. For practitioners in climate science and high-impact weather prediction, these findings underscore the need to evaluate individual model behaviors against reanalysis data instead of assuming a blanket superiority or inferiority based on the AI label.
Do AI weather models miss extremes?
arXiv:2608.09972v2 Announce Type: replace-cross Abstract: AI weather models are often reported to underestimate extremes, but most evidence concerns deterministic regression models verified against reanalysis. We evaluate twelve physical and AI forecast models against ECMWF IFS using ten months of European station observations. The evaluation covers 10 m wind, 2 m temperature, solar radiation, and precipitation within regimes defined from a fixed ERA5 1991-2020 climatology. We find no uniform AI-specific deficit in the tails. Several AI models remain more accurate than IFS under extreme conditions, while others deteriorate markedly; comparable variation occurs among physical models. Every model nevertheless exhibits a common conditional-error pattern, overpredicting low observations and underpredicting high observations. Attenuation of extreme values therefore does not imply a uniform loss of relative skill: tail performance depends on the model, variable, and evaluation setting rather than on whether the forecast is produced by AI or physical numerical modelling.
Title:Do AI weather models miss extremes?
Abstract:AI weather models are often reported to underestimate extremes, but most evidence concerns deterministic regression models verified against reanalysis. We evaluate twelve physical and AI forecast models against ECMWF IFS using ten months of European station observations. The evaluation covers 10 m wind, 2 m temperature, solar radiation, and precipitation within regimes defined from a fixed ERA5 1991-2020 climatology. We find no uniform AI-specific deficit in the tails. Several AI models remain more accurate than IFS under extreme conditions, while others deteriorate markedly; comparable variation occurs among physical models. Every model nevertheless exhibits a common conditional-error pattern, overpredicting low observations and underpredicting high observations. Attenuation of extreme values therefore does not imply a uniform loss of relative skill: tail performance depends on the model, variable, and evaluation setting rather than on whether the forecast is produced by AI or physical numerical modelling.
Atmospheric and Oceanic Physics (physics.ao-ph); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2608.09972 [physics.ao-ph]
(or arXiv:2608.09972v2 [physics.ao-ph] for this version)
https://doi.org/10.48550/arXiv.2608.09972 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 248 words
Related coverage
UniGIO: Unified Generative Global In-situ Weather Modeling from Spatiotemporal Incomplete Observations
arXiv · Sep 22, 2026
Presentation: Postgres for Production Agents: Your Relational Foundation for Enterprise AI
InfoQ · Jul 15, 2026
Introducing WeatherNext 3, our most advanced and accurate global weather AI model
Google DeepMind · Sep 3, 2026
Evaluating and improving crop-yield forecasting methods during extreme drought
arXiv · Aug 20, 2026