[3E1-GS-2] Machine learning

[3E1-GS-2-04] Model Calibration Using Expectation and Variance of Decision Loss

〇Kohei Munechika¹, Satoshi Hara¹ (1. Osaka University)

Keywords:Calibration, Decision Making

Classification models produce probabilities for each class as a measure of confidence in their predictions.
Calibration is a technique used to adjust these confidence levels in order to better align them with the actual data.
This is particularly important for high-performance models like deep neural networks, which may produce confidence levels that differ from reality.
Decision Calibration (DC) is a type of calibration method that uses a user's expected loss (decision loss) when making a decision to calibrate the model.
When selecting a model, it is important to consider not only the expectation of the decision loss, but also its variance.
In this study, we propose a calibration method taking both the expectation and variance of the decision loss into consideration.