5:15 PM - 6:30 PM
[ACG43-P04] Real-time Data Quality Control of Phased Array Weather Radar by using Semantic Segmentation
Keywords:Phased Array Weather Radar, Semantic Segmentation, Quality Control
The observation data of Phased Array Weather Radar (PAWR) which performs volume scan every 30 seconds is used for real-time precipitation prediction such as 3D precipitation nowcast (Otsuka, et al., 2016. doi: 10.1175 / WAF-D-15-0063.1) and the smartphone application "3D Amagumo Weather". The data quality control (QC) of PAWR is necessary to make accurate predictions, and It is important to remove non-precipitation echoes (clutters derived from buildings on the ground and mountains, fake echoes appear before and after strong echoes which is called range side-lobe echoes). In the current real-time QC, clutter discrimination is performed using thresholds of the Doppler velocity and the texture information of the reflectivity. However, if the data QC is performed in various cases with a certain threshold value, erroneous judgment may occur. Therefore, to perform appropriate QC in various cases, we apply semantic segmentation, one of the deep learning applications that detects objects from images in pixel units.
We report the results using ESPNet (Mehta, et al., 2018.arXiv: 1803.06815v3), which has high inferences speed. The data was selected from 10 cases of convective rainfall observed by PAWR installed at the Osaka University from 2015 to 2017, and a total of 666 images with elevation angles 6 to 12 (corresponding to elevation angles 4.4 ° to 9.4 °) were prepared. For stratiform rainfall, another 10 cases were selected, and 644 images were prepared. By using color images of reflectivity and annotation images (0 = no data, 1 = precipitation data, 2 = non-precipitation data) as training data, two models for convection and stratiform rainfall were created. For the annotations, the QC flag created by the conventional method was used, and the flag data that was considered to be false judgments by visual inspection was removed as much as possible.
Figure 1 shows the inference results of convective (left) and stratiform rainfall (right) with an elevation angle of 6.2 ° converted to PPI in the Cartesian coordinates. They are examples of input data (reflectivity: upper row), inference results (middle row), and the QC flag (lower row: as ground truth). In the result of inferences and the QC flag, black part is judged to be a non-precipitation echo. In the inference of convective rainfall, the range side-lobe echoes are discriminated, and the judgment is very similar to the QC flag. Looking at the inference results of stratiform rainfall, the non-precipitation echoes were hardly identified on this elevation angle, and the model inference was performed correctly.
In this study, learning and inference were performed on a Linux server equipped with one GeForce RXT 2080 Ti. For convective rainfall, it took about 0.02 seconds to inference the data for one elevation angle, and for stratiform precipitation with a large precipitation area, it took about 0.04 seconds. The reading of observation data was accelerated by calling a C program using Cython (0.035 seconds / file, 1 file is about 40MB). Considering the data processing of 110 elevation angles obtained by the PAWR observation every 30 seconds, it was expected that QC processing would be performed in a few seconds.
As a future work, it is necessary to create other models for each elevation angles, because the types of non-precipitation echoes in the data near lower elevation and zenith angles are different from the above. In order to further improve the accuracy of inference, we think that it is necessary to tune the QC flag used as annotations for each case, create more correct ground truth, and increase the number of learning data. The goal is to complete a processing system that can process data reading, convection / stratiform judgment by CNN, segmentation of total elevation angles data, aggregation and output of inferences results within 10 seconds.
This work was supported by JST AIP Grant Number JPMJCR19U2, Japan.
We report the results using ESPNet (Mehta, et al., 2018.arXiv: 1803.06815v3), which has high inferences speed. The data was selected from 10 cases of convective rainfall observed by PAWR installed at the Osaka University from 2015 to 2017, and a total of 666 images with elevation angles 6 to 12 (corresponding to elevation angles 4.4 ° to 9.4 °) were prepared. For stratiform rainfall, another 10 cases were selected, and 644 images were prepared. By using color images of reflectivity and annotation images (0 = no data, 1 = precipitation data, 2 = non-precipitation data) as training data, two models for convection and stratiform rainfall were created. For the annotations, the QC flag created by the conventional method was used, and the flag data that was considered to be false judgments by visual inspection was removed as much as possible.
Figure 1 shows the inference results of convective (left) and stratiform rainfall (right) with an elevation angle of 6.2 ° converted to PPI in the Cartesian coordinates. They are examples of input data (reflectivity: upper row), inference results (middle row), and the QC flag (lower row: as ground truth). In the result of inferences and the QC flag, black part is judged to be a non-precipitation echo. In the inference of convective rainfall, the range side-lobe echoes are discriminated, and the judgment is very similar to the QC flag. Looking at the inference results of stratiform rainfall, the non-precipitation echoes were hardly identified on this elevation angle, and the model inference was performed correctly.
In this study, learning and inference were performed on a Linux server equipped with one GeForce RXT 2080 Ti. For convective rainfall, it took about 0.02 seconds to inference the data for one elevation angle, and for stratiform precipitation with a large precipitation area, it took about 0.04 seconds. The reading of observation data was accelerated by calling a C program using Cython (0.035 seconds / file, 1 file is about 40MB). Considering the data processing of 110 elevation angles obtained by the PAWR observation every 30 seconds, it was expected that QC processing would be performed in a few seconds.
As a future work, it is necessary to create other models for each elevation angles, because the types of non-precipitation echoes in the data near lower elevation and zenith angles are different from the above. In order to further improve the accuracy of inference, we think that it is necessary to tune the QC flag used as annotations for each case, create more correct ground truth, and increase the number of learning data. The goal is to complete a processing system that can process data reading, convection / stratiform judgment by CNN, segmentation of total elevation angles data, aggregation and output of inferences results within 10 seconds.
This work was supported by JST AIP Grant Number JPMJCR19U2, Japan.