[2C5-GS-2] Machine learning: reinforcement learning (2)

[2C5-GS-2-05] A Study on Online Learning Based on Bayesian Optimization to Suppress Deterioration of Objective Function Value

〇Yuka Nakamura¹, Taiga Yoshikawa¹, Ayako Yamagiwa¹, Masayuki Goto¹ (1. Waseda University)

Keywords:Online Learning, Gaussian process regression, Bandit problem, recommendation, exploration

In recent years, many recommender systems used in e-commerce sites estimate preferring items for each user based on the past log data, and show them as a list. The performance of these systems is evaluated by measuring whether presented recommendation lists match the preferences of customers. However, the recommendation is not implemented just once, but is in fact a continuous process. Thus, it is important to discuss the performance based on the cumulative loss for the entire recommendation series.
Meanwhile, online learning is a framework that can handle such sequential recommendation and evaluation. However, the purpose of online learning is to improve the efficiency of learning in order to estimate condition of best values, and most methods do not consider the cumulative loss of recommendations. On the other hand, Safe Exploration for Optimization(SafeOpt) has recently been proposed as a method to perform exploration while suppressing the deterioration of the objective function. However, this method has a problem that the searchable range depends on the initial input.
In this study, we extend the original SafeOpt by introducing GP-UCB, which is a method for global search, and propose a method to suppress the deterioration of the objective function for a wide search area.
Finally, we show the effectiveness of the proposed method by generating artificial data and conducting experiments.