[2M5-OS-24] OS-24

[2M5-OS-24-04] Neural Architecture Search for Multiple Architectures with Different Model Size Complexities

〇Toshiyuki Nishimoto¹, Shota Saito^1,2, Yoichi Hirose¹, Kento Uchida¹, Shinichi Shirakawa¹ (1. Yokohama National University, 2. SKILLUP NeXt Ltd.)

Keywords:Neural Architecture Search, Constrained Black-Box Optimization, Convolutional Neural Network, Probabilistic Model-based Optimization

For the utilization of deep neural networks in low-resource devices, several neural architecture search (NAS) methods search the architectures of deep neural networks that realize superior predictive performance under resource constraints, such as model size and latency. Moreover, increasing the variety of low-resource devices requires NAS methods to obtain multiple models under different constraints. This study proposes a NAS method for searching multiple architectures satisfying different model size constraints simultaneously. We update multiple categorical distributions following the stochastic relaxation technique with importance sampling. This update uses a few architectures generated from the mixture distribution, which reduces the search cost. Additionally, we introduce the penalty function method with coefficient adaptation to obtain architectures satisfying the model size constraints. We also introduce a sampling strategy to obtain architectures satisfying the constraints from the optimized distribution. The experimental results show that the proposed method achieved architectures with higher predictive performance under constraints.