9:45 AM - 10:00 AM
[SSS14-04] Development of the damage detection method for buildings with machine learning techniques utilizing aerial photographs of the Kumamoto earthquake
Keywords:Deep Learning, SVM, Machine Learning, Aerial Photograph, Building Damage, Kumamoto Earthquake
NIED has been developing the Real-time Damage Estimation and Identification System for use in supporting decision-makings for emergency responses. As a part of the project, we are researching about the damage detection method utilizing an aerial photographs for aftermath of an earthquake. The image acquisition with an airplane is superior at immediacy and widespread photographing. Until now, the damage extraction from images have been performed with visual judgement. On the other hand, in case of next coming huge disasters like the Nankai Trough earthquake, it is not considered practical because it would take long terms and huge human resources. So, in this study, we are developing the automatic damage identification method with aerial photos utilizing machine learning techniques.
First, we acquired orthographic projection photographs taken at April 19th that is soon after the main-shock of 2016 Kumamoto earthquake, in adjacent to damage-concentrated areas such as Mashiki-town. With these images, we classified damage of buildings into 4 levels, then input damage grades and building types into GIS polygons. Furthermore, with these GIS data, extracted images of more than 10,000 building which have equally damage levels of 2,500 buildings, then prepared mask images which was color-coded in in each polygons according as different damage levels.
We developed discriminant models by two ways. Model.1: a model using the feature extraction and the Support Vector Machine (SVM), Model.2: a model with the Convolutional Neural Network (CNN).
In case of model.1, raster scan in the entire image, and calculate SIFT features as a 256-dimensional vector sequence with 2 different radii of each characteristic points. Next, divide feature vectors into 1,000 groups as a centroid of each clusters with k-means clustering method, then search nearest neighbor clusters in each characteristic points and make histograms of appearance frequency. Finally, classify these features with non-linear Support Vector Machine.
In case of model.2, we developed the CNN referred to VGG. CNN extract features with image patches by combination of Convolutional layers which apply filters to the images and Pooling layers which compress images. Besides, we added procedures for enhancing the performance such as shuffling training data, sorting mini-batches, Batch Normalization and Drop Out.
We have tested each discrimination models by use of the aerial photo which size is 1.5km×2km that represents seriously damaged area. In case of model.1, at first, raster scan entire the image in increments of 64 pixels square patches, then divide each patches into 5 classes. In case of model.2, at first, extract 80 pixels square patches which represent features of classified damage levels from the entire image. The number of extracting patches are 2,500 in each 5 classes. As results of 10-hold cross validation, accuracy rate of each models are approximately 74% (model.1), and 92% (model.2). So, the Deep Learning show superiority in discrimination accuracy. Furthermore, in both case, computational time was within 10 minutes, under conditions of different computational environment that model.1 uses CPU and model.2 uses GPU. So, both models show massively promptness compared to the human judgement which takes 1 day to discriminate 1,000 buildings.
By the way, for emergency response, it is required not only to indicate the distribution of the damaged area but also to quantify the number of damaged buildings. So, we are developing the damage classification method for each buildings based on the threshold according as the area ratio of the discriminant result patches.
This work was supported by Cross-ministerial Strategic Innovation Promotion Program (SIP), titled “Enhancement of societal resiliency against natural disasters”.
First, we acquired orthographic projection photographs taken at April 19th that is soon after the main-shock of 2016 Kumamoto earthquake, in adjacent to damage-concentrated areas such as Mashiki-town. With these images, we classified damage of buildings into 4 levels, then input damage grades and building types into GIS polygons. Furthermore, with these GIS data, extracted images of more than 10,000 building which have equally damage levels of 2,500 buildings, then prepared mask images which was color-coded in in each polygons according as different damage levels.
We developed discriminant models by two ways. Model.1: a model using the feature extraction and the Support Vector Machine (SVM), Model.2: a model with the Convolutional Neural Network (CNN).
In case of model.1, raster scan in the entire image, and calculate SIFT features as a 256-dimensional vector sequence with 2 different radii of each characteristic points. Next, divide feature vectors into 1,000 groups as a centroid of each clusters with k-means clustering method, then search nearest neighbor clusters in each characteristic points and make histograms of appearance frequency. Finally, classify these features with non-linear Support Vector Machine.
In case of model.2, we developed the CNN referred to VGG. CNN extract features with image patches by combination of Convolutional layers which apply filters to the images and Pooling layers which compress images. Besides, we added procedures for enhancing the performance such as shuffling training data, sorting mini-batches, Batch Normalization and Drop Out.
We have tested each discrimination models by use of the aerial photo which size is 1.5km×2km that represents seriously damaged area. In case of model.1, at first, raster scan entire the image in increments of 64 pixels square patches, then divide each patches into 5 classes. In case of model.2, at first, extract 80 pixels square patches which represent features of classified damage levels from the entire image. The number of extracting patches are 2,500 in each 5 classes. As results of 10-hold cross validation, accuracy rate of each models are approximately 74% (model.1), and 92% (model.2). So, the Deep Learning show superiority in discrimination accuracy. Furthermore, in both case, computational time was within 10 minutes, under conditions of different computational environment that model.1 uses CPU and model.2 uses GPU. So, both models show massively promptness compared to the human judgement which takes 1 day to discriminate 1,000 buildings.
By the way, for emergency response, it is required not only to indicate the distribution of the damaged area but also to quantify the number of damaged buildings. So, we are developing the damage classification method for each buildings based on the threshold according as the area ratio of the discriminant result patches.
This work was supported by Cross-ministerial Strategic Innovation Promotion Program (SIP), titled “Enhancement of societal resiliency against natural disasters”.