[M-GI27] Data-driven approaches for weather and hydrological predictions

[MGI27-11] Toward enhancing the ensemble Kalman filter with a diffusion model

*Takumi Honda¹, Yohei Sawada² (1.Information Technology Center, The University of Tokyo, 2.Graduate School of Engineering, The University of Tokyo)

Keywords:Data assimilation, Machine learning, Numerical weather prediction, Diffusion model

Data assimilation provides improved initial conditions by combining information from observations and simulations taking their uncertainties into account. The ensemble Kalman filter (EnKF), a major data assimilation method, leverages differences within an ensemble of simulations to represent forecast uncertainty. To accurately estimate forecast uncertainty, a large ensemble is necessary, but this can be computationally expensive. Recent studies have indicated that diffusion models, a type of generative artificial intelligence (AI), can generate a large ensemble at relatively low computational cost. This study aims to integrate a diffusion model into an EnKF system to improve state estimation accuracy. Specifically, we use a diffusion model to generate a large ensemble using a small conventional ensemble simulation as input. By doing so, we mitigate sampling errors due to limited ensemble sizes. As a proof-of-concept, this study focuses on a low-dimensional chaotic system. We built a diffusion model based on a U-net neural network and trained it on a small conventional ensemble dataset. Preliminary results indicate that our diffusion model successfully generates a large ensemble that captures the general characteristics of the forecast error covariance. Next, we will evaluate the performance of an EnKF system coupled with the trained diffusion model.