2:00 PM - 2:15 PM
[MAG39-02] Extraction method of weather informations from camera image using Deep Neural Network
Keywords:Deep Learning, IoT, High-density Weather Observations
In order to further improve precision of numerical forecasts, it is expected to utilize high-density weather information by various IoT (Internet of Things) sensor equipment networks capable of real-time acquisition via the Internet. However, the IoT sensors are installed on the ground, so we can not observe weather information in the sky simply by using the IoT sensor normally. In this study, we used artificial intelligence technology for image recognition, and worked on the development of a system that extracts cloud amount and solar radiation from images of inexpensive consumer cameras. Our method becomes possible to operate low-price camera-equipped devices as remote sensing equipment without using expensive observation equipment. The weather information extracted is not only input values for numerical simulation but also expected to be utilized in, for example, agriculture. In addition, the society of security camera network is spreading in China and others. Our method can be a fundamental technology of high-density weather observation in such society.
In this study, we first constructed a 5-layer convolutional neural network (CNN) model that predicts cloud cover amount from the image. As data to train this network, we used data labeled cloud cover on about 2,000 sky images from the Internet. The cloud cover in this study means the rate of the clouds in the sky area shown in the camera image in 11 steps from 0 to 10. On the other hand, the Japan Meteorological Agency defines the rate of the cloud region to the whole sky as the cloud amount. Attention should be paid to difference in this definition. We used the 80% data as the training data, and the remaining 20% was used for cross validation. Finally, we compared to the MSE: mean square error, of our method to the existing method. It became clear that cloud cover can be estimated with the overwhelmingly small MSE of 3.0 for this CNN, more than the existing segmentation method of 12.7.
Secondly, as training data, this model use long-term high-frequency observation data for 1 minute, observed by Yokohama National University. We chose solar radiation amount, fixed camera image in this data. The model extracts the amount of solar radiation from the sky image in the range of several tens of kilometers ahead of the fixed camera's photographing direction. With 6 years of this data, the number of images reaches max about 3 million images in one direction, and this big training data is used to train Deep Neural Network. Now, intuitively, a high brightness image tends to be considered to have a high solar radiation amount. However, consumer cameras usually have automatic brightness and contrast adjustment functions. Also, such as the slightly cloudy days are almost high brightness value. The correlation coefficient of brightness value and solar radiation, is not large. On the other hand, we can calculate the theoretical maximum solar radiation from the latitude, longitude and time of image's shooting position (Nakajima et al. 2000). Focusing on this point, we constructed a new Multimodal Deep Learning model with image + theoretical maximum solar radiation as input layers. This model estimates the attenuation value from theoretical maximum solar radiation with images of training data. It was clarified that the amount of solar radiation can be estimated with high accuracy by training this model from the image, position and time.
In this study, we first constructed a 5-layer convolutional neural network (CNN) model that predicts cloud cover amount from the image. As data to train this network, we used data labeled cloud cover on about 2,000 sky images from the Internet. The cloud cover in this study means the rate of the clouds in the sky area shown in the camera image in 11 steps from 0 to 10. On the other hand, the Japan Meteorological Agency defines the rate of the cloud region to the whole sky as the cloud amount. Attention should be paid to difference in this definition. We used the 80% data as the training data, and the remaining 20% was used for cross validation. Finally, we compared to the MSE: mean square error, of our method to the existing method. It became clear that cloud cover can be estimated with the overwhelmingly small MSE of 3.0 for this CNN, more than the existing segmentation method of 12.7.
Secondly, as training data, this model use long-term high-frequency observation data for 1 minute, observed by Yokohama National University. We chose solar radiation amount, fixed camera image in this data. The model extracts the amount of solar radiation from the sky image in the range of several tens of kilometers ahead of the fixed camera's photographing direction. With 6 years of this data, the number of images reaches max about 3 million images in one direction, and this big training data is used to train Deep Neural Network. Now, intuitively, a high brightness image tends to be considered to have a high solar radiation amount. However, consumer cameras usually have automatic brightness and contrast adjustment functions. Also, such as the slightly cloudy days are almost high brightness value. The correlation coefficient of brightness value and solar radiation, is not large. On the other hand, we can calculate the theoretical maximum solar radiation from the latitude, longitude and time of image's shooting position (Nakajima et al. 2000). Focusing on this point, we constructed a new Multimodal Deep Learning model with image + theoretical maximum solar radiation as input layers. This model estimates the attenuation value from theoretical maximum solar radiation with images of training data. It was clarified that the amount of solar radiation can be estimated with high accuracy by training this model from the image, position and time.