[3J3-OS-3a] AutoML（自動機械学習）(1/2)

[3J3-OS-3a-01] Local Search with Multi start for Hyperparameter Optimization of Deep Learning

〇Shintaro Takenaga¹, Masaki Onishi² (1. University of Tsukuba, 2. National Institute of Advanced Industrial Science and Technology)

Keywords:Hyperparameter Optimization, Local Search, Deep Learning

In deep learning, hyperparameters can severely affect the learning model performance. Hyperparameter optimization (HPO) is one of the promising techniques to maximize the performance of the learning model. It has been reported that the Nelder-Mead method shows superior performance to other optimization methods in the HPO of deep learning. However, the Nelder-Mead method may converge bad local minima because this method is a local search heuristic using a simplex. This problem may be tackled using multi start to start from the different initial values. In this paper, we investigate the effectiveness of the multi start in several HPO problems of deep learning. The results show that the search performance of the Nelder-Mead method is improved by applying the multi start.