2:05 PM - 2:20 PM
[MAG32-02] Utilization of Deep Learning in mapping of the ocean floor: Extraction of brittle stars by image recognition, seagrass distribution using image to image translation
Keywords:Seagrass bed, Benthic organisms, Machine Learning, Geographical Information System (GIS), Big data, 2011 Tōhoku earthquake and tsunami
Recently the development of pattern recognition techniques has undergone a dramatic rate increase. The application of such systems is expected to enable easier and/or more accurate image processing. Here we explain two different approaches to recognize objects in images and discuss their usefullness for mapping the ocean floor.
For the first approach, we introduce the use of object recognition to map the distribution of benthic organisms following the impacts of the Great East Japan Earthquake, which occurred on 11 March 2011. To assess the broad-scale impact of the earthquake along the continental shelf and slope it is necessary to extract occurrence data for organisms observed during biological surveys.
Such data is normally produced by counting organisms occurring in images manually, and therefore takes a long time to produce. Here we applied classical object recognition methods with Haar-like features and Deep Learning methods to extract various objects in images or videos.
Although there are several popular, pre-existing natural image datasets that can be used as training data for such purposes, they cannot classify correctly any objects that do not already occur in the image dataset. Thus, we first prepared an image dataset of marine organisms from the Tohoku area. This dataset was mainly comprised of the dominant species of Ophiuroidea (brittle stars) for use as training data for automatic recognition (Yamakita et al. 2018 and Yamakita 2018).
Using this data set we produced a training set containing 11368 images of Ophiuroidea and 5264 other images. The results, using Haar-like features, were an 82% rate of for correct detection (sensitivity) and a 12% rate for incorrect detection (false positive rate), when compared with human eye/manual identification in 635 images. We introduce the results of the Deep Learning approach in this presentation.
For the second approach, we introduce results obtained using image to image transformation with Conditional Adversarial Nets (cGAN). One such application for this is image generation (e.g. using pix2pix software), which is a technique used to produce an output image with similar characteristics to an input image and which could enable us to produce maps or land cover information automatically from aerial photographs.
We chose seagrass bed extraction for the case study. Seagrass bed areas are highly productive and considered to be nursery areas for fishes, but they are generally thought to be declining world-wide. Here we investigated the long-term dynamics of seagrass beds in the relatively natural seagrass bed in a marine protected area, Hat Chao Mai National Park, Trang Province, Thailand. This area of South East Asia is a well-known data gap with regards to long-term information. Remote sensing is one of the best ways to observe long-term dynamics in shallow water seagrass beds. We were able to collect aerial photographs and satellite image data beginning in the 1970s, and we tried to assess the effect that the advent of Deep Learning technology can have on such remote sensing applications. To that purpose, we applied the above-mentioned recent techniques to the application of monitoring seagrass beds and compared image classification methods including pixel based/object-based supervised classification (semi-automatic) and Deep Learning (automatic classification).
The accuracy of the classification was 84%+-6.48% for semi-automatic classification and 89%+-6.6 for automatic classification without consideration of the density class (two classes: mostly sand or mostly seagrass). Evaluation including two classes of density (projection coverage) was 55%+-15.8% and 63%+-4.48, respectively. Because semi-automatic classification normally requires training data for each image (which takes a long time to produce), automatic classification using a pretrained model provides a quick (within 10's of seconds) method of extraction. However, some challenges, such as artefacts observed at the edges of the analysis units, remain.
One result concerning the dynamics of the seagrass beds, especially in the shallow areas close to river mouths, was that they varied greatly depending on sand and channel movement. We also show an example from the Futtsu tidal flat, Tokyo Bay, Japan, which we previously analyzed using data over a 30-year period (Yamakita et al. 2005, 2011).
(references are at the end of the Japanese abstract)
For the first approach, we introduce the use of object recognition to map the distribution of benthic organisms following the impacts of the Great East Japan Earthquake, which occurred on 11 March 2011. To assess the broad-scale impact of the earthquake along the continental shelf and slope it is necessary to extract occurrence data for organisms observed during biological surveys.
Such data is normally produced by counting organisms occurring in images manually, and therefore takes a long time to produce. Here we applied classical object recognition methods with Haar-like features and Deep Learning methods to extract various objects in images or videos.
Although there are several popular, pre-existing natural image datasets that can be used as training data for such purposes, they cannot classify correctly any objects that do not already occur in the image dataset. Thus, we first prepared an image dataset of marine organisms from the Tohoku area. This dataset was mainly comprised of the dominant species of Ophiuroidea (brittle stars) for use as training data for automatic recognition (Yamakita et al. 2018 and Yamakita 2018).
Using this data set we produced a training set containing 11368 images of Ophiuroidea and 5264 other images. The results, using Haar-like features, were an 82% rate of for correct detection (sensitivity) and a 12% rate for incorrect detection (false positive rate), when compared with human eye/manual identification in 635 images. We introduce the results of the Deep Learning approach in this presentation.
For the second approach, we introduce results obtained using image to image transformation with Conditional Adversarial Nets (cGAN). One such application for this is image generation (e.g. using pix2pix software), which is a technique used to produce an output image with similar characteristics to an input image and which could enable us to produce maps or land cover information automatically from aerial photographs.
We chose seagrass bed extraction for the case study. Seagrass bed areas are highly productive and considered to be nursery areas for fishes, but they are generally thought to be declining world-wide. Here we investigated the long-term dynamics of seagrass beds in the relatively natural seagrass bed in a marine protected area, Hat Chao Mai National Park, Trang Province, Thailand. This area of South East Asia is a well-known data gap with regards to long-term information. Remote sensing is one of the best ways to observe long-term dynamics in shallow water seagrass beds. We were able to collect aerial photographs and satellite image data beginning in the 1970s, and we tried to assess the effect that the advent of Deep Learning technology can have on such remote sensing applications. To that purpose, we applied the above-mentioned recent techniques to the application of monitoring seagrass beds and compared image classification methods including pixel based/object-based supervised classification (semi-automatic) and Deep Learning (automatic classification).
The accuracy of the classification was 84%+-6.48% for semi-automatic classification and 89%+-6.6 for automatic classification without consideration of the density class (two classes: mostly sand or mostly seagrass). Evaluation including two classes of density (projection coverage) was 55%+-15.8% and 63%+-4.48, respectively. Because semi-automatic classification normally requires training data for each image (which takes a long time to produce), automatic classification using a pretrained model provides a quick (within 10's of seconds) method of extraction. However, some challenges, such as artefacts observed at the edges of the analysis units, remain.
One result concerning the dynamics of the seagrass beds, especially in the shallow areas close to river mouths, was that they varied greatly depending on sand and channel movement. We also show an example from the Futtsu tidal flat, Tokyo Bay, Japan, which we previously analyzed using data over a 30-year period (Yamakita et al. 2005, 2011).
(references are at the end of the Japanese abstract)