5:15 PM - 6:45 PM
[STT38-P06] Hessian-based uncertainty quantification for Physics-Informed Neural Networks: Application to the frictional parameter estimation
Keywords:Physics-Informed Neural Networks (PINNs), uncertainty quantification, Hessian matrix, rate and state dependent friction law, earthquake cycle simulation
Physics-Informed Neural Networks (PINNs) are new machine-learning techniques for solving partial differential equations and determining the controlling parameters of the equations using observation (Raissi et al., 2019). PINNs have been successfully applied to various inverse problems in seismology, including full-waveform inversion (Rasht-Behesht et al., 2022) and seismic tomography (Waheed et al., 2021). Fukushima et al. (JpGU 2024) applied PINNs for estimating the spatial distribution of frictional parameters from synthetic GNSS observations of slow slip events. In such inverse problems, uncertainty quantification is crucial for evaluating the reliability of the estimated values. For this purpose, Bayesian Physics-Informed Neural Networks (B-PINNs) have been proposed to evaluate the uncertainties by considering neural network parameters as stochastic variables (Yang et al., 2021). This formulation enables us to calculate the posterior distribution of neural network parameters or estimated physical parameters through the Hamiltonian Monte Carlo method (HMC) or the variational inference (VI). However, due to the large computation cost for evaluating the posterior distribution, the application of B-PINNs to geophysics problems is still limited (e.g. Agata et al., 2023).
Ito et al. (2022) proposed the Hessian-based uncertainty quantification method by employing the second-order adjoint method and conducted uncertainty quantification for the heterogeneous frictional parameter estimation. Inspired by this study, we propose a simple uncertainty quantification method for PINNs based on the Hessian matrix. The Hessian matrix, consisting of the second derivation of loss functions with respect to neural network parameters, can be easily calculated through automatic differentiation in the PINNs’ framework. We approximate the posterior distribution as a multivariate Gaussian distribution around the optimized values by employing the Hessian matrix, and sample neural network parameters from this Gaussian distribution. Sampling from multivariate Gaussian distribution can be easily conducted by the appropriate linear transformation corresponding to the directions of the eigenvectors of the Hessian matrix. Then, by repeating the forward calculation from the sampled neural network parameters, we calculate the variance of the function represented by the neural network. This method allows us to quantify the uncertainties in the estimated parameters with extremely low computation costs compared to HMC or VI. It should be noted that this method cannot evaluate potentially existing multiple peaks of the posterior distribution, since it evaluates the uncertainty just around the optimized values. We apply the proposed method for the estimation of frictional parameter distribution (Fukushima et al., JpGU 2024) and discuss the uncertainty of estimated frictional parameters with different observation data.
Ito et al. (2022) proposed the Hessian-based uncertainty quantification method by employing the second-order adjoint method and conducted uncertainty quantification for the heterogeneous frictional parameter estimation. Inspired by this study, we propose a simple uncertainty quantification method for PINNs based on the Hessian matrix. The Hessian matrix, consisting of the second derivation of loss functions with respect to neural network parameters, can be easily calculated through automatic differentiation in the PINNs’ framework. We approximate the posterior distribution as a multivariate Gaussian distribution around the optimized values by employing the Hessian matrix, and sample neural network parameters from this Gaussian distribution. Sampling from multivariate Gaussian distribution can be easily conducted by the appropriate linear transformation corresponding to the directions of the eigenvectors of the Hessian matrix. Then, by repeating the forward calculation from the sampled neural network parameters, we calculate the variance of the function represented by the neural network. This method allows us to quantify the uncertainties in the estimated parameters with extremely low computation costs compared to HMC or VI. It should be noted that this method cannot evaluate potentially existing multiple peaks of the posterior distribution, since it evaluates the uncertainty just around the optimized values. We apply the proposed method for the estimation of frictional parameter distribution (Fukushima et al., JpGU 2024) and discuss the uncertainty of estimated frictional parameters with different observation data.