[H-DS27_1AM2] Tsunami and its Forecast

[HDS27-12] Proper scoring systems with definite connections to information values of tsunami warnings

*Yutaka HAYASHI¹ (1.Meteorological Research Institute)

Keywords:binary forecast, cost-loss model, expected-utility theory, rational decision-making, score

Necessary conditions which newly introduced method can improve forecast are, existing proper scoring system, and that the new method marks better score than the present method do. Up to now, these scoring system have never applied to tsunami warning system.Some scoring rules being applied widely to binary forecasts in weather forecasting, such as having precipitation or not, have close connections to change of utility for users. These scores are based on assumption that all user know their cost to make counter measures (C) and loss in case of no counter measure (L). When the forecast says the event will occur, and all users are assumed to make counter measures. In addition, a simple probability density distribution of U(-C)/U(-L) is assumed for cost-loss model, where U is the utility function. In general, a score is calculated by using a targeted dataset, e.g., a fixed period of time, and frequencies: occurrence of targeted phenomena is forecasted and observed (hit: N_a), forecasted but not occurred (false alarm N_), not forecasted but occurred (misdetection: N_c), and not forecasted and not occurred (hit: N_d). For example, equitable threat score (ETS≡(N_a-K)/(N_a+N_+N_c-K), where K≡(N_a+N_)(N_a+N_c)/(N_a+N_+N_c+N_d) ) is one of their scoring system.In this paper, suitable scoring rules for tsunami warnings are derived by considering the characteristics of tsunami warnings and following assumptions.(1) Scores can be defined without N_d, because counting N_d does not make sense for tsunami warning.(2) In case of tsunami warning, users of forecasts can select actions to take a counter measure or not. In case of no warning, users do not take a counter measure. Change of utilities are U(-C) and U(-L) for taking a counter measure and for when a phenomenon happens without a counter measure, respectively.(3) All users know the fault alarm ratio (FAR≡N_/(N_a+N_)) of the warning, their utilities for each condition (U(-C), U(-L)), and then their rational decision-making choose the option so that their expectation of utility (E_x(U)) become maximum. Here, if U(-L)/U(-C) < FAR/(1-FAR) is satisfied, not taking a counter measure is the more reasonable decision. According to this assumption, larger the FAR is, larger the cost-loss ratio is, warning become easier to be ignored.(4) Assuming three types of probability density functions on x=U(-C)/U(-L). a) Uniform model: f(x)=1, b) Low-cost model: f(x)=2-2x, and (c) High-cost model: f(x)=2x for the range of 0≤x≤1.(5) The scores are set to be proportional to the information value of the warning. Here, △U can be calculated as the integral corresponding to each distribution of (4) and utilities of selected actions at the N_a+N_ warnings based on the rational decision-making described in (3). Besides, if there were not for warning system, users should have lose utility as much as -U(-L) at every event. Then, V≡-ΔU/((N_a+N_c)U(-L)).Scores corresponding to models a)-c) in (4) are derived as follows.a) V=N_a²/(2(N_a+N_)(N_a+N_c)). For good warning which satisfies both N_a≫N_ and N_a≫N_c, the score can be approximated to V≒CSI/2, where CSI≡N_a/(N_a+N_+N_c) is threat score or critical success index.b) V=(2/3)(1-FAR)(1-M)(1+M/2), where M≡N_c/(N_a+N_c) is missing ratio. For warnings with few misdetection which satisfies Nc≪Na, the score can be approximated to be V≒(2/3)(1-FAR)(1-M/2).(c) V=(1-FAR)²(1-M)/3.The proper score system thus changes according to the cost-loss ratio, which have close relation to preparedness. It is necessary to choose suitable forecast method using proper scoring system which is corresponding to a social structure. In the meeting, the author would like to discuss also on the problem for the practical application of the scoring systems.