Presentation information

Organized Session

Organized Session » OS-3

[3J4-OS-3b] AutoML(自動機械学習)(2/2)

Thu. Jun 16, 2022 3:30 PM - 5:10 PM Room J (Room J)

オーガナイザ:大西 正輝(産業技術総合研究所)[現地]、日野 英逸(統計数理研究所/理化学研究所)

3:30 PM - 3:50 PM

[3J4-OS-3b-01] Stopping criterion for Neural Architecture Search

〇Kotaro Sakamoto Sakamoto1, Hideaki Ishibashi2, Rei Sato3, Shinichi Shirakawa4, Yohei Akimoto3,5, Hideitsu Hino1,5 (1. The Institute of Statistical Mathematics, 2. Kyushu Institute of Technology, 3. University of Tsukuba, 4. Yokohama National University, 5. RIKEN Center for Advanced Intelligence Project (AIP))

Keywords:Neural Architecture Search, Deep Learning

Neural architecture search (NAS) is a framework for automating the design process of a neural network structure. While the recent one-shot approaches have reduced the search cost, there still exists an inherent trade-off between cost and performance. It is important to appropriately stop the search and further minimise the high cost of NAS. On the other hand, heuristic early-stopping strategies have been proposed to overcome the well-known performance degradation of the one-shot approach, particularly differentiable architecture search (DARTS). In this paper, we propose a more versatile and principled early-stopping criterion on the basis of the evaluation of a gap between expectation values of generalisation errors of the previous and current search steps with respect to the architecture parameters. The stopping threshold is automatically determined at each search epoch without cost. In numerical experiments, we demonstrate the effectiveness of the proposed method. We stop the one-shot NAS algorithms such as ASNG-NAS and DARTS and evaluate the acquired architectures on the benchmark datasets: NAS-Bench-201 and NATS-Bench. Our algorithm has been shown to reduce the cost of the search process while maintaining a high performance.

Authentication for paper PDF access

A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.