[S-SS06] Seismic wave propagation: Theory and Application

[SSS06-P01] Theoretical seismic waveform calculation with an accuracy of 1.6 seconds by the spectral element method using the Earth Simulator (ES4)

*Seiji Tsuboi¹, Rhett Butler², Tsuyoshi Nakagawa¹ (1.JAMSTEC, Center for Earth Information Science and Technology, 2.University of Hawaii at Manoa)

Keywords:Theoretical seismic wave, Spectral element method, Structure of the Earth’s interior

We have performed global seismic wave propagation calculations using the spectral element method, which is a type of finite element method, for a realistic earth model. In 2016, we used the K computer's 82,134 nodes (99% of all nodes) by dividing the earth model into 665.2 billion grid points to perform theoretical seismic waveform recording calculations with an accuracy of about 1.2 seconds. (Tsuboi et al., 2016). This time, we report that the same scale of calculation was performed by the Earth Simulator (ES4) system, which started operation in March 2021. In the calculation of the spectral element method, the division of the earth model divides the entire earth into six quadrangular pyramids, and each quadrangular pyramid is divided into finer quadrangular pyramids and assigned to individual CPUs of the supercomputer to perform the calculation. In this calculation, the theoretical seismic waveform propagating globally with an accuracy of 1.6 seconds was calculated by dividing it into 244.7 billion grid points. The parameters NEX and NPROC indicating the division of the spectral element method in this case are 2656 and 83, respectively, and the total number of cores used in the calculation is 41,334 and the ES4 vector engine (VE) is 5168. The grid point spacing in this mesh is 0.94 km on average. For this scale of calculation, it took about 30 minutes CPU time to calculate the mesh and 4 hours 40 minutes CPU time to calculate the theoretical seismic waveform for 23 minutes. The size of the mesh is about 41 Tbyte. The calculation used NEC's MPI as a flat MPI, and the effective performance according to the MPI Program Information was 1.13 PFLOPS, and the vectorization rate was 99%. This effective performance is about 8.8% of the theoretical peak performance of 5168VE. The Earth's internal structure model used in the calculation is transversely isotropic PREM (Dziewonski and Anderson, 1981) for the radial symmetric structure model, and s362ani (Kustowski et al., 2006) for the mantle three-dimensional structure. In addition, the ellipsoidal shape is adopted and attenuation is taken into consideration, but the effects of gravity and rotation are not taken into consideration. The combination of the earthquake and the observation station was targeted at the observation waveform at the earthquake and its antipodal point, which was dealt with in Butler and Tsuboi (2010, 2020, 2021). In particular, we calculated the records at the Qiongzhong station in China due to the April 17, 2009 earthquake in northern Chile (Ms6.1). This is to examine the structure that reproduces the Cdiff phase discussed in Butler and Tsuboi (2010). The model used and the calculated theoretical seismic waveform will be described at the time of presentation.
Acknowledgment: We used the Earth Simlutaor operated by the Japan Agency for Marine-Earth Science and Technology. We used the program package SPECFEM3D_GLOBE for spectral element method.