[AAS01-10] Toward an integrated NWP-DA-AI system for precipitation prediction
Keywords:nowcast, machine learning, NWP
The Phased-Array Weather Radar (PAWR), developed by the National Institute of Information and Communications Technology, Osaka University, and Toshiba Corporation, has been in operations since 2012 in Japan. The PAWR scans the whole sky in the 60-km range every 30 seconds at 110 elevation angles. Four PAWRs of the same type have been installed in Osaka, Kobe, Okinawa, and Tsukuba, and two similar ones of other types have been installed in Japan. Taking advantage of the PAWRs' frequent and dense three-dimensional volume scans, we developed two systems: a high-resolution regional numerical weather prediction (NWP) system (SCALE-LETKF, Miyoshi et al., 2016a,b, Lien et al., 2017), and a three-dimensional (3D) precipitation nowcasting system (Otsuka et al., 2016).
Our 30-second-update 3D nowcasting system is running in real time since 2017; the system adopts an optical-flow-based algorithm in the 3D space. Because convective clouds evolve rapidly within a 10-minute forecast, sometimes the assumption of Lagrangian persistence is violated, and the prediction skill of the optical-flow-based system drops quickly with the forecast lead time. The SCALE-LETKF system, on the other hand, provides physically based predictions; therefore, we would expect that the NWP outperforms the nowcast for longer forecasts. Therefore, merging NWP and nowcast will provide better predictions compared to each of them.
Recent advances in the machine-learning algorithms will provide an efficient algorithm for that purpose. In this study, a three-dimensional extension of the Convolutional Long Short-Term Memory (Conv-LSTM; Shi et al., 2015), a kind of deep-learning algorithm, is applied to PAWR nowcasting. In addition to the Conv-LSTM with past observations, we also develop a Conv-LSTM that accepts forecast data from numerical weather prediction (NWP) or optical-flow-based nowcast. NWP uses HPC resources with full physics equations of the atmosphere, so that Conv-LSTM with NWP would be a new direction toward fusing Big Data and HPC, in which training with the big data from high-resolution NWP and Data Assimilation (DA), as well as PAWR observation would be a challenge.
The 3D Conv-LSTM successfully made predictions of convective storms; in some cases, Conv-LSTM had additional skill in capturing intensification and weakening of precipitation that were not predicted by the optical-flow. On average, the Conv-LSTM-based system outperformed the optical-flow-based system statistically. Furthermore, Conv-LSTM with forecast data outperformed that without forecast data.
Our 30-second-update 3D nowcasting system is running in real time since 2017; the system adopts an optical-flow-based algorithm in the 3D space. Because convective clouds evolve rapidly within a 10-minute forecast, sometimes the assumption of Lagrangian persistence is violated, and the prediction skill of the optical-flow-based system drops quickly with the forecast lead time. The SCALE-LETKF system, on the other hand, provides physically based predictions; therefore, we would expect that the NWP outperforms the nowcast for longer forecasts. Therefore, merging NWP and nowcast will provide better predictions compared to each of them.
Recent advances in the machine-learning algorithms will provide an efficient algorithm for that purpose. In this study, a three-dimensional extension of the Convolutional Long Short-Term Memory (Conv-LSTM; Shi et al., 2015), a kind of deep-learning algorithm, is applied to PAWR nowcasting. In addition to the Conv-LSTM with past observations, we also develop a Conv-LSTM that accepts forecast data from numerical weather prediction (NWP) or optical-flow-based nowcast. NWP uses HPC resources with full physics equations of the atmosphere, so that Conv-LSTM with NWP would be a new direction toward fusing Big Data and HPC, in which training with the big data from high-resolution NWP and Data Assimilation (DA), as well as PAWR observation would be a challenge.
The 3D Conv-LSTM successfully made predictions of convective storms; in some cases, Conv-LSTM had additional skill in capturing intensification and weakening of precipitation that were not predicted by the optical-flow. On average, the Conv-LSTM-based system outperformed the optical-flow-based system statistically. Furthermore, Conv-LSTM with forecast data outperformed that without forecast data.