aiqbench analysis
This study provides a critical assessment of machine learning weather prediction models in the complex terrain of Northern Norway, comparing their performance against traditional numerical weather prediction methods. The findings highlight that while MLWP models like FourCastNet3 and GraphCast are now competitive with high-resolution forecasts for local wind speed, they still struggle to accurately capture extreme wind events in fjord environments. This research is significant for AI practitioners working on geospatial modeling, as it underscores the necessity of refining architectures to better handle complex topography before these tools can be fully trusted in operational settings.
A Station-Based Evaluation of Machine Learning-based Weather Forecasting Models in Northern Norway
arXiv:2609.10564v1 Announce Type: cross Abstract: Recent machine learning weather prediction (MLWP) models have demonstrated remarkable forecasting skill on global reanalysis-based benchmarks. However, their performance remains unclear in challenging environments such as Northern Norway, where narrow fjords and rapidly changing weather result in highly variable local wind conditions. In this case study, we evaluate FourCastNet3 (FCN3), GraphCast, and ECMWF High Resolution Forecast (HRES) for wind speed forecasting using multi-year station observations from Northern Norway, focusing on their relative performance, generalization beyond the training period, and performance under high-wind conditions. Our results show that HRES slightly outperforms FCN3 and GraphCast, with an overall RMSE of 2.89 $\mathrm{m\,s^{-1}}$, compared to 2.96 $\mathrm{m\,s^{-1}}$ for FCN3 and 2.94 $\mathrm{m\,s^{-1}}$ for GraphCast. Notably, the MLWP models maintain comparable performance beyond their respective training periods, with no clear evidence of noticeable degradation. FCN3 performs best under high-wind conditions, although all models substantially underestimate strong winds. Our findings suggest that MLWP has become competitive with NWP for local wind, but further refinements are still needed to capture complex terrain better.
Abstract:Recent machine learning weather prediction (MLWP) models have demonstrated remarkable forecasting skill on global reanalysis-based benchmarks. However, their performance remains unclear in challenging environments such as Northern Norway, where narrow fjords and rapidly changing weather result in highly variable local wind conditions. In this case study, we evaluate FourCastNet3 (FCN3), GraphCast, and ECMWF High Resolution Forecast (HRES) for wind speed forecasting using multi-year station observations from Northern Norway, focusing on their relative performance, generalization beyond the training period, and performance under high-wind conditions. Our results show that HRES slightly outperforms FCN3 and GraphCast, with an overall RMSE of 2.89 $\mathrm{m\,s^{-1}}$, compared to 2.96 $\mathrm{m\,s^{-1}}$ for FCN3 and 2.94 $\mathrm{m\,s^{-1}}$ for GraphCast. Notably, the MLWP models maintain comparable performance beyond their respective training periods, with no clear evidence of noticeable degradation. FCN3 performs best under high-wind conditions, although all models substantially underestimate strong winds. Our findings suggest that MLWP has become competitive with NWP for local wind, but further refinements are still needed to capture complex terrain better.
Atmospheric and Oceanic Physics (physics.ao-ph); Machine Learning (cs.LG)
arXiv:2609.10564 [physics.ao-ph]
(or arXiv:2609.10564v1 [physics.ao-ph] for this version)
https://doi.org/10.48550/arXiv.2609.10564 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 264 words
Related coverage
Statistical versus machine learning-based spatial interpolation of post-processed ensemble weather forecasts
arXiv · Sep 9, 2026
Enhancing Strawberry Yield Forecasting with Backcasted IoT Sensor Data and Machine Learning
arXiv · Jun 9, 2026
News
Quantum Computing Report · Jul 25, 2026
Observation-driven correction of numerical weather prediction for marine winds
arXiv · Jun 9, 2026