[P-EM12] Space Weather and Space Climate

[PEM12-P24] Why do some probabilistic forecasts lack reliability?

*Yuki Kubo¹ (1.National Institute of Information and Communications Technology)

Keywords:Probabilistic forecast, Reliability

One of the important attributes to be satisfied for the probabilistic forecast system is reliability. However, some probabilistic forecast systems lack reliability. Why do some probabilistic forecasts lack reliability? We investigate a condition for a probabilistic binary forecast to be reliable in this work. We mathematically prove that a necessary, but not sufficient, condition for achieving a reliable probabilistic forecast is maximizing the Peirce skill score (PSS) at the threshold probability of the climatological base rate. The condition is confirmed by using artificially synthesized forecast-outcome pair data and previously published probabilistic solar flare forecast models. The condition gives a partial answer as to why some probabilistic forecast systems lack reliability, because the system, which does not satisfy the proved condition, can never be reliable. The result implies that those who want to develop a reliable probabilistic forecast system must adjust or train the system so as to maximize PSS near the threshold probability of the climatological base rate.