11:00 AM - 11:15 AM
[MGI26-07] A Theory of Variational Lower Bound for Data Assimilation and Its Application Using Variational Autoencoders
Keywords:Data Assimilation, Variational Bayes, Variational Autoencoder, Super-Resolution, Neural Network, Deep Learning
Data assimilation is a framework that estimates the state of a system based on observations and background information given by a model. This framework can be formulated using Bayes’ theorem. Recently, a new variational Bayesian method [1] has been proposed by employing variational autoencoders (VAEs), which include deep Kalman filters [2]. This method has mainly been applied to natural language processing and has not been utilized in data assimilation. This study formulates a variational lower bound (VLB) for data assimilation and demonstrates the simultaneous feasibility of assimilation and super-resolution (downscaling) by training a VAE with the proposed VLB.
The new VLB is composed of two terms: reconstruction loss and Kullback-Leibler divergence. In the VLB maximization, the first (second) term becomes larger as the estimated state approaches the observation (background). Thus, the VLB-maximum estimator is interpreted as a generalization of minimum-variance estimators. There is a degree of freedom for the prior distribution in the VLB. We provide this prior with a neural network that super-resolves the background state. By exploiting this freedom, data assimilation and super-resolution can be achieved simultaneously. A VAE can then be constructed via unsupervised learning using the VLB as a loss function.
The proposed method was validated using an idealized two-dimensional oceanic jet stream [3]. The true state was obtained from high-resolution (128x64) simulations. The synthetic observation was generated by subsampling the true state (2.7% grid points) and adding Gaussian noise. The background state was obtained from low-resolution (32x16) simulations. The length of the assimilation cycles was approximately half of the advection time scale. An ensemble Kalman filter (EnKF) was employed for comparison. To make the EnKF inference more accurate, the low-resolution background state was mapped to the high-resolution grid points using bicubic interpolation and the EnKF was then applied in this high-resolution space [4].
Figure shows vorticity snapshots from the analysis, along with the time series of errors obtained from all test simulations. Without data assimilation or super-resolution, labeled by "Without DA or SR", the vorticity field was significantly different from the true state. Both the EnKF and VAE successfully estimated the large-scale pattern of the true state, whereas the vortex shapes given by the VAE were finer than those by the EnKF. Even without ensemble evolution, the grid-point wise error of the VAE was comparable to that of the EnKF. Furthermore, the spatial-pattern error of the VAE was smaller than that of the EnKF, suggesting that the super-resolution can infer spatial patterns similar to those of the true state. The average wall time of simulations using the VAE was approximately 25 seconds, which is about one-eighth of that of the EnKF, 216 seconds.
[1] Girin et al. (2021), doi: 10.1561/2200000089
[2] Krishnan et al. (2015), doi: 10.48550/ARXIV.1511.05121
[3] David et al. (2017), doi: 10.1016/j.ocemod.2017.03.008
[4] Barthelemy et al. (2022), doi: 10.1007/s10236-022-01523-x
[5] Wang et al. (2004), doi: 10.1109/TIP.2003.819861
The new VLB is composed of two terms: reconstruction loss and Kullback-Leibler divergence. In the VLB maximization, the first (second) term becomes larger as the estimated state approaches the observation (background). Thus, the VLB-maximum estimator is interpreted as a generalization of minimum-variance estimators. There is a degree of freedom for the prior distribution in the VLB. We provide this prior with a neural network that super-resolves the background state. By exploiting this freedom, data assimilation and super-resolution can be achieved simultaneously. A VAE can then be constructed via unsupervised learning using the VLB as a loss function.
The proposed method was validated using an idealized two-dimensional oceanic jet stream [3]. The true state was obtained from high-resolution (128x64) simulations. The synthetic observation was generated by subsampling the true state (2.7% grid points) and adding Gaussian noise. The background state was obtained from low-resolution (32x16) simulations. The length of the assimilation cycles was approximately half of the advection time scale. An ensemble Kalman filter (EnKF) was employed for comparison. To make the EnKF inference more accurate, the low-resolution background state was mapped to the high-resolution grid points using bicubic interpolation and the EnKF was then applied in this high-resolution space [4].
Figure shows vorticity snapshots from the analysis, along with the time series of errors obtained from all test simulations. Without data assimilation or super-resolution, labeled by "Without DA or SR", the vorticity field was significantly different from the true state. Both the EnKF and VAE successfully estimated the large-scale pattern of the true state, whereas the vortex shapes given by the VAE were finer than those by the EnKF. Even without ensemble evolution, the grid-point wise error of the VAE was comparable to that of the EnKF. Furthermore, the spatial-pattern error of the VAE was smaller than that of the EnKF, suggesting that the super-resolution can infer spatial patterns similar to those of the true state. The average wall time of simulations using the VAE was approximately 25 seconds, which is about one-eighth of that of the EnKF, 216 seconds.
[1] Girin et al. (2021), doi: 10.1561/2200000089
[2] Krishnan et al. (2015), doi: 10.48550/ARXIV.1511.05121
[3] David et al. (2017), doi: 10.1016/j.ocemod.2017.03.008
[4] Barthelemy et al. (2022), doi: 10.1007/s10236-022-01523-x
[5] Wang et al. (2004), doi: 10.1109/TIP.2003.819861