[2T4-OS-5a] OS-5

[2T4-OS-5a-02] Automatic Generation of Datasets with High Evaluation Capability for Machine Learning Algorithms Using Item Response Theory

〇Takeaki Sakabe¹, Yuko Sakurai¹, Emiko Tsutsumi², Satoshi Oyama^3,4 (1. Nagoya Institute of Technology, 2. The University of Tokyo, 3. Nagoya City University, 4. RIKEN)

Keywords:Item Response Theory, Conditional Variational Autoencoder, Data Analysis Competition

We propose a framework for generating datasets that can appropriately evaluate the performance of the proposed algorithms in competitions. In most competitions which students and engineers studying machine learning participate in, the dataset is selected ad hoc. Therefore, there has been an issue, such as the use of dataset that would yield high performance no matter what algorithm is used. To resolve these problems, we conbine Item Response Theory and Conditional VAE. Item Response Theory is a theory for creating test questions and evaluating examinees' abilities. Conditional VAE is a method for generating images according to parameters. Experimental results show that our method generates a dataset which can evaluate the performance of algorithms appropriately more than MNIST.