9:15 AM - 9:30 AM
[MGI33-02] Current status of a galaxy formation simulation code for Fugaku
Keywords:Galaxy formation, Numerical simulation
We are now developing a galaxy formation simulation code ASURA-FDPS, which is fully tuned for the Fugaku supercomputer. In this presentation, we will report the current status of the ASURA-FDPS, following the presentation in JpGU 2020.
Even in the current state-of-the-art simulations of galaxy formation, it is difficult to handle individual stars in these simulations due to limitations of computational resources and numerical techniques. Therefore, the simple stellar population (SSP) approximation where star particles in simulation consist of stars with the same age and metallicity under a given initial mass function is used. With this, the evolution of stars such as energy feedbacks and chemical evolution are taken into account. On the other hand, the recent process of observations is remarkable. The Gaia satellite provides the phase-space information of billions of stars in the Milkyway galaxy and the upcoming Subaru-PFS will extend it to the nearby galaxies. Thus, there is a growing demand for theoretical models which can directly compare with observations. Our ASURA-FDPS is designed to solve the evolution of individual stars during galaxy formation by combining the computational power of the Fugaku supercomputer and a framework for particle simulations with massively-parallel supercomputers, FDPS.
In 2020, we were concentrated on the tuning of the gravity and hydrodynamics routines. In Mar. 2021, we achieved 31 Pflops using the whole system of Fugaku for the computation of both gravity and hydrodynamics. The use of the whole system of Fugaku provided us with a lot of insights to use it. ASURA-FDPS adopts the hybrid parallel computation using MPI and OpenMP but because of the limitations of the physical memory, say 32 GB/node, we could not use 4 MPI processes and 16 OpenMP threads for each MPI process in a computational node and we hampered to use 1 MPI process and 48 OpenMP threads. This reduced the computational performance by almost half. In addition, we figured out that a large amount of memory is required in order to use MPI comm split. We now developing the code with the fully shared step method instead of our Hamiltonian split integration method.
As well as these performance data, we will give the current state of our galaxy formation simulations in our presentation.
Even in the current state-of-the-art simulations of galaxy formation, it is difficult to handle individual stars in these simulations due to limitations of computational resources and numerical techniques. Therefore, the simple stellar population (SSP) approximation where star particles in simulation consist of stars with the same age and metallicity under a given initial mass function is used. With this, the evolution of stars such as energy feedbacks and chemical evolution are taken into account. On the other hand, the recent process of observations is remarkable. The Gaia satellite provides the phase-space information of billions of stars in the Milkyway galaxy and the upcoming Subaru-PFS will extend it to the nearby galaxies. Thus, there is a growing demand for theoretical models which can directly compare with observations. Our ASURA-FDPS is designed to solve the evolution of individual stars during galaxy formation by combining the computational power of the Fugaku supercomputer and a framework for particle simulations with massively-parallel supercomputers, FDPS.
In 2020, we were concentrated on the tuning of the gravity and hydrodynamics routines. In Mar. 2021, we achieved 31 Pflops using the whole system of Fugaku for the computation of both gravity and hydrodynamics. The use of the whole system of Fugaku provided us with a lot of insights to use it. ASURA-FDPS adopts the hybrid parallel computation using MPI and OpenMP but because of the limitations of the physical memory, say 32 GB/node, we could not use 4 MPI processes and 16 OpenMP threads for each MPI process in a computational node and we hampered to use 1 MPI process and 48 OpenMP threads. This reduced the computational performance by almost half. In addition, we figured out that a large amount of memory is required in order to use MPI comm split. We now developing the code with the fully shared step method instead of our Hamiltonian split integration method.
As well as these performance data, we will give the current state of our galaxy formation simulations in our presentation.