[3D5-GS-2] Machine learning

[3D5-GS-2-01] Performance comparison of multiple models with a small amount of labeling

〇Mitsuru Matsuura¹, Satoshi Hara¹ (1. Osaka University)

Keywords:Active learning, Performance comparison

Supervised learning typically requires a large amount of data, and the cost of labeling can be significant if
obtaining labels is difficult. This is also true when evaluating a model. Active testing is introduced to address this
challenge by estimating the average test loss through actively labeling a portion of test data. However, in practical
machine learning, multiple good model candidates are often available instead of just one. The problem of interest
then becomes selecting the best performing model. In this study, we propose a method to compare the performance
of multiple models with a small amount of labeling by extending the framework of active testing.