[M-IS09] Ocean Plastics, an earth science perspective

[MIS09-09] Demonstration experiment for the quantification of plastic litter in the city and automatic generation of the training data for litter detection using the 3DCG model

*Ryunosuke Muroya¹, Koyo Ijima², Shinichiro Kako^1,3, Daisuke Matsuoka^1,3, Atsuhiko Isobe⁴, So Sasaki⁵, Hideo Sakurai⁶, Yasushi Ikebe⁶ (1.Graduate School of Science and Engineering, Kagoshima University, 2.Fasulty of Engineering, Kagoshima University, 3.Japan Agency for Marine-Earth Science and Technology, 4.Research Institute for Applied Mechanics, Kyushu University, 5.Chuo University, 6.ITABASHI Science and Education Hall)

Keywords:Street litter, Citizen science, Smartphone application, Deep learning, Object detection, 3DCG

Many marine plastic litter is domestic waste that has been discharged from cities due to improper management and has entered the ocean via rivers (Lebreton et al., 2017). However, quantification method for plastic litter in cities based on objective analysis and the efficient observation method is not yet established on global scale. Therefore, we proposed a method for quantifying plastic litter in cities using images obtained through citizen science using a smartphone app “Pirika” and image analysis based deep leaning (Muroya et al., JpGU2023). In this study, we conducted a collaborated demonstration experiment with citizens with the method using Pirika to quantify litter in cities. Additionally, aiming for higher versatility and more detailed classification, we explored the automatic construction method of the litter images for training data. The demonstration experiment was organized by the ITABASHI Science & Education HALL. Participants collected litter images using “Pirika” while picking up litter along a predefined route based on a pre survey. Since participants collected and photographed all the litter along the route, it is possible to estimate the current amount of litter in the city from the starting point to the endpoint by counting the litter in the images. Upon visually confirming the images collected during this demonstration experiment, we found many types of litter (e.g., food packaging such as bread and sweets) that were not included in the deep learning model we constructed. This result indicates that it is essential to construct a deep learning model that can detect and classify a greater variety of street litter to achieve the quantification of plastic litter in cities using our method. To construct a model that can detect the images of the target litter in cities, we have to collect a lot of data sufficient for training of the image analysis. Therefore, we attempted to construct a system for automatically generating the images for training data using 3-Dimensional Computer Graphics (3DCG) models created with Blender. 3DCG allows for the easy production of many images by setting the color, background, shooting distance, and angle, and so forth. When creating training data, it is necessary to apply annotations that extract the class classification (what is depicted in the image) and pixel coordinates of the class target object in the image. To do this annotation process, we used Roboflow (https://roboflow.com), which was done manually. With Roboflow, it is possible to automate augmentation (data enhancement) of training data and the annotation processes using models constructed on Roboflow or public modes (e.g., MSCOCO). In this study, we constructed the various type of deep learning based image analysis methods by changing the training data. By comparing these models with that of Muroya et al. (JpGU2023), we discussed the practicality and challenges of the automatic construction method of the images of litter in cities.