aiqbench analysis
This study provides a critical global benchmark comparing AI weather prediction models like GraphCast and AIFS against traditional numerical systems for forecasting atmospheric river precipitation. While AI models demonstrate superior spatial skill and pattern preservation over longer lead times, they exhibit distinct biases in predicting extreme rainfall magnitudes compared to NWP systems. These findings highlight the current limitations of AI in capturing the intensity of hydrometeorological hazards despite advances in large-scale variable prediction.
Global Evaluation of AI and NWP Precipitation Forecasts During Atmospheric River Events
arXiv:2610.03758v1 Announce Type: cross Abstract: Atmospheric rivers (ARs) produce many of the world's most extreme precipitation events and hydrometeorological hazards. Although artificial intelligence weather prediction (AIWP) models have demonstrated skill comparable to or exceeding numerical weather prediction (NWP) systems for large-scale atmospheric variables, their ability to forecast AR-related precipitation remains insufficiently characterized globally. Here, we evaluate 24-hour precipitation forecasts from the Global Forecast System (GFS), Global Ensemble Forecast System (GEFS), GraphCast, and Artificial Intelligence Forecasting System (AIFS) from Day 1 through Day 10 globally and across North America, Europe, and Australia and New Zealand. Using the Extreme Weather Bench framework, forecasts are evaluated against Integrated Multi-satellitE Retrievals for GPM (IMERG) observations using measures of precipitation magnitude, spatial structure, and localization. GraphCast and AIFS exhibit greater spatial skill than GFS and GEFS, particularly for heavy precipitation and at longer lead times, and better preserve the spatial organization of AR-related precipitation through Day 10. However, this improved spatial skill does not translate into accurate precipitation magnitudes. AIWP models tend to overpredict moderate-to-heavy accumulations while underpredicting the heaviest precipitation at longer lead times, whereas NWP systems develop pronounced dry biases. These results reveal distinct strengths and limitations of AIWP for high-impact precipitation forecasting and provide a reproducible benchmark.
Abstract:Atmospheric rivers (ARs) produce many of the world's most extreme precipitation events and hydrometeorological hazards. Although artificial intelligence weather prediction (AIWP) models have demonstrated skill comparable to or exceeding numerical weather prediction (NWP) systems for large-scale atmospheric variables, their ability to forecast AR-related precipitation remains insufficiently characterized globally. Here, we evaluate 24-hour precipitation forecasts from the Global Forecast System (GFS), Global Ensemble Forecast System (GEFS), GraphCast, and Artificial Intelligence Forecasting System (AIFS) from Day 1 through Day 10 globally and across North America, Europe, and Australia and New Zealand. Using the Extreme Weather Bench framework, forecasts are evaluated against Integrated Multi-satellitE Retrievals for GPM (IMERG) observations using measures of precipitation magnitude, spatial structure, and localization. GraphCast and AIFS exhibit greater spatial skill than GFS and GEFS, particularly for heavy precipitation and at longer lead times, and better preserve the spatial organization of AR-related precipitation through Day 10. However, this improved spatial skill does not translate into accurate precipitation magnitudes. AIWP models tend to overpredict moderate-to-heavy accumulations while underpredicting the heaviest precipitation at longer lead times, whereas NWP systems develop pronounced dry biases. These results reveal distinct strengths and limitations of AIWP for high-impact precipitation forecasting and provide a reproducible benchmark.
Atmospheric and Oceanic Physics (physics.ao-ph); Artificial Intelligence (cs.AI)
arXiv:2610.03758 [physics.ao-ph]
(or arXiv:2610.03758v1 [physics.ao-ph] for this version)
https://doi.org/10.48550/arXiv.2610.03758 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 297 words
Related coverage
Diffusion Precipitation Generation with Medium-range Weather Forecasting
arXiv · Oct 12, 2026
Introducing WeatherNext 3, our most advanced and accurate global weather AI model
Google DeepMind · Sep 3, 2026
ORBIT-2 based Weather and Climate Downscaling and Downscaled Global Forecasts on AMD Instinct
AMD · Jun 8, 2026
Physics-Guided Dual Decoding and Spectral Supervision for Global 3D Hydrometeor Prediction
arXiv · Jun 9, 2026