[1J3-J-2] Machine learning: bayesian models

[1J3-J-2-02] Scalable Bayesian Optimization with Memory Retention

〇Hidetaka Ito¹, Tatsushi Matsubayashi¹, Takeshi Kurashima¹, Hiroyuki Toda¹ (1. NTT Service Evolution Laboratories, NTT Corporation)

Keywords:Bayesian Optimization, Gaussian Process

Bayesian optimization is a method for the global optimization of black-box functions as few evaluation as possible. It utilizes Gaussian processes to efficiently select parameters to be evaluated. However, it is not scalable because Gaussian processes scale cubically with the number of iterations. In this work, we propose a method for scalable Bayesian optimization by leveraging models used in past iterations, which we call past memory. This technique enables us to t Gaussian processes to only input-output pairs near the previously selected input parameter. In experiments, we show our proposed method outperforms naive Bayesian optimization in terms of optimization performance with limited time budget.