Presentation information

Organized Session

Organized Session » OS-10

[1J3-OS-10] OS-10

Tue. Jun 9, 2020 1:20 PM - 2:40 PM Room J (jsai2020online-10)

大西 正輝(産業技術総合研究所)、日野 英逸(統計数理研究所)、白川 真一(横浜国立大学)、秋本 洋平(筑波大学)

1:40 PM - 2:00 PM

[1J3-OS-10-02] The Characteristics Required in Hyperparameter Optimization of Deep Learning Algorithms

〇Shuhei Watanabe1, Masahiro Nomura2, Masaki Onishi1 (1. The National Institute of Advanced Industrial Science and Technology, 2. CyberAgent Inc.)

Keywords:Hyperparameter Optimization, Black-box Optimization, Deep Learning

Since the performance of deep learning algorithms depends seriously on the selection of hyperparameter values, hyperparameter optimization is essential in real-world applications.
Many researchers have studied hyperparameter optimization algorithms extensively.
However, there has been not enough discussion on the characteristics of the relationship, which we call objective function, between the searching space and its performance in the context of hyperparameter optimization.
Therefore, we elucidate the properties that the objective functions in this paper.
More specifically, we evaluated hyperparameter settings of deep learning algorithms with 5 hyperparameters.
Then, a regression model for each objective function was constructed to analyze the objective function.
Finally, we reached the conclusion that hyperparameter methods should be able to deal with ill-conditioned first and then consider non-separable and less important hyperparameters and convexity is a strong assumption on hyperparameter optimization.

Authentication for paper PDF access

A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.