Presentation information

General Session

General Session » [GS] J-2 Machine learning

[3K3-J-2] Machine learning: analysis and validations of models

Thu. Jun 6, 2019 1:50 PM - 3:30 PM Room K (201A Medium meeting room)

Chair:Masahiro Suzuki Reviewer:Satoshi Oyama

2:30 PM - 2:50 PM

[3K3-J-2-03] Do the AUC and log-loss evaluate CTR prediction models properly?

〇Satoshi KATAGIRI1 (1. F@N Communications, Inc.)

Keywords:Machine Learning, CTR Prediction, Evaluation Metrics, Calibration, Online Advertising

Click-through rate (CTR) prediction is one of the most important task for web advertising platform companies. However, CTR prediction is a non-standard machine learning task, so conventional metrics, for example, area under the Receiver Operating Characteristic curve (AUC), and log-loss, a.k.a. cross-entropy, and so on, can be improper. Our target is develop a new metrices for CTR prediction. In this article, we state the drawbacks of such conventional metrics and perspective of a metric based on the calibration plot approach.