aiqbench analysis
This study demonstrates that reinforcement learning can effectively replace static, offline-tuned parameters in weather models with dynamic, state-dependent functions to reduce persistent biases. By testing across idealized climate testbeds, the research identifies specific RL algorithms like DDPG and TD3 as superior for stabilizing model drift while maintaining physical consistency in radiative processes. The findings suggest a scalable pathway for integrating online learning directly into numerical weather prediction frameworks without sacrificing interpretability or stability.
Replacing Tunable Parameters in Weather and Climate Models with State-Dependent Functions using Reinforcement Learning
arXiv:2601.04268v3 Announce Type: replace Abstract: Weather and climate models rely on parametrisations to represent unresolved sub-grid processes. Traditional schemes rely on fixed coefficients that are weakly constrained and tuned offline, contributing to persistent biases that limit their ability to adapt to underlying physics. This study presents a framework that learns components of parametrisation schemes online as a function of the evolving model state using reinforcement learning (RL) and evaluates policy-driven parameter updates across idealised testbeds spanning a simple climate bias correction (SCBC), a radiative-convective equilibrium (RCE), and a zonal mean energy balance model (EBM) with single-agent and federated multi-agent settings. Across nine RL algorithms, Truncated Quantile Critics (TQC), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed DDPG (TD3) achieved the highest skill and stable convergence, with performance assessed against a static baseline using area-weighted RMSE, temperature and pressure-level diagnostics. For the EBM, single-agent RL outperformed static parameter tuning with the strongest gains in tropical and mid-latitude bands, while federated RL on multi-agent setups enabled specialised control and faster convergence, with a six-agent DDPG configuration using frequent aggregation yielding the lowest area-weighted RMSE across the tropics and mid-latitudes. The learnt corrections were also physically meaningful as agents modulated EBM radiative parameters to reduce meridional biases, adjusted RCE lapse rates to match vertical temperature errors, and stabilised heating increments to limit drift. Overall, results show that RL can learn skilful state-dependent parametrisation components in idealised settings, offering a scalable pathway for online learning within numerical models and a starting point for evaluation in weather and climate models.
Abstract:Weather and climate models rely on parametrisations to represent unresolved sub-grid processes. Traditional schemes rely on fixed coefficients that are weakly constrained and tuned offline, contributing to persistent biases that limit their ability to adapt to underlying physics. This study presents a framework that learns components of parametrisation schemes online as a function of the evolving model state using reinforcement learning (RL) and evaluates policy-driven parameter updates across idealised testbeds spanning a simple climate bias correction (SCBC), a radiative-convective equilibrium (RCE), and a zonal mean energy balance model (EBM) with single-agent and federated multi-agent settings. Across nine RL algorithms, Truncated Quantile Critics (TQC), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed DDPG (TD3) achieved the highest skill and stable convergence, with performance assessed against a static baseline using area-weighted RMSE, temperature and pressure-level diagnostics. For the EBM, single-agent RL outperformed static parameter tuning with the strongest gains in tropical and mid-latitude bands, while federated RL on multi-agent setups enabled specialised control and faster convergence, with a six-agent DDPG configuration using frequent aggregation yielding the lowest area-weighted RMSE across the tropics and mid-latitudes. The learnt corrections were also physically meaningful as agents modulated EBM radiative parameters to reduce meridional biases, adjusted RCE lapse rates to match vertical temperature errors, and stabilised heating increments to limit drift. Overall, results show that RL can learn skilful state-dependent parametrisation components in idealised settings, offering a scalable pathway for online learning within numerical models and a starting point for evaluation in weather and climate models.
Machine Learning (cs.LG); Atmospheric and Oceanic Physics (physics.ao-ph)
arXiv:2601.04268 [cs.LG]
(or arXiv:2601.04268v3 [cs.LG] for this version)
https://doi.org/10.1029/2026MS005745 DOI(s) linking to related resources
Full text under the publisher's open licence · 272 words
Related coverage
ORBIT-2 based Weather and Climate Downscaling and Downscaled Global Forecasts on AMD Instinct
AMD · Jun 8, 2026
2026.10: You are here 📍
Home Assistant · Oct 7, 2026
Deconfusing Corrigibility
AI Alignment Forum · Oct 17, 2026
Aurora 1.5: Extending open foundation models for weather and Earth-system applications
Microsoft Research · Jul 9, 2026