[S-SS10] Strong Ground Motion and Earthquake Disaster

[SSS10-P21] Development of a Low-cost Seismic behavior prediction system for furniture and nonstructural components using RGB-D Camera and Machine Learning

*Yuta Takahashi¹, Hiroki UEMATSU¹, Ahyi KIM¹, Toshiyuki MASATSUKI² (1.Yokohama City University, 2.KOZO KEIKAKU ENGINEERING Inc.)

Earthquake hazard maps provide information on the hazards around the home during an earthquake, which is very useful for individual disaster prevention and mitigation measures. However, it is difficult to know the hazards inside the home, as it is strongly influenced by the conditions inside and outside the building. One method of knowing indoor hazards during an earthquake is to simulate the real space by replacing it with a 3D model. However, such modeling is expensive and time-consuming, so it is not easy to simulate room of an ordinary home. One idea to solve this problem is to automate the whole process from 3D model creation to simulation. Therefore, in this research, we aim to develop a system to monitor the condition of the building and disaster information, and to predict the response of the room to earthquake motion, by combining a relatively inexpensive IoT MEMS sensor that can be installed in the house and RGB-D. Specifically, we will attach the RGB-D camera to the IoT MEMS sensor on the Raspberry Pi, detect objects, make 3D models of them, simulate the shaking in response to earthquake motion, and build a system to predict the danger lurking in the room. In order to predict hazards, we need to create 3D models of furniture and other room structures. To do so, we will 1) photograph the room using an RGB-D camera with a depth sensor that captures RGB images and measures their distance. (2) The RGB-D camera captures point cloud data from the RGB image and its depth information to create a 3D model of the room. (3) Identify objects by machine learning using RGB images captured by the RGB-D camera and images of the 3D model taken from various angles, and assign physical coefficients such as mass and friction coefficient to the 3D model based on the size of the furniture. To identify the objects, we fine-tune the existing object detection trained model by machine learning to suit the application of this research. The behavior of furniture in the room is simulated by inputting seismic motion to the model created in this way. We prepare several scenario earthquakes as the input earthquake motion for the simulation, and add the response characteristics of the building extracted from the earthquake waveforms recorded by the IoT MEMS sensors installed inside the building and on the ground surface of the building site. The response characteristics will be updated every time the IoT MEMS sensor experiences an earthquake, which will contribute to improving the accuracy of the simulation. The accuracy of the simulation is also expected to be improved by updating the physical coefficients of the furniture based on the images of the room conditions before and after the earthquake. In this presentation, the data obtained from the room excitation experiment of E-defense will be modeled into a 3D model using the above method, and the simulation will be performed to evaluate its performance in comparison with the actual experimental results.