[3Xin4] Poster session 1

[3Xin4-28] Extended Bayesian Inference in the Multi-armed Bandit Problem

〇DAIKI MORITA¹, Nobuhito Manome², Tatsuji Takahashi¹, Shuji Shinohara^1,2 (1.Tokyo Denki University, 2.Graduate School of Engineering, The University of Tokyo)

Keywords:Reinforcement Learning, Bayesian Inference, Bandit problem

Bayesian inference is a statistical inference method that can probabilistically infer the process of data generation from observed data, and is one of the fundamental technologies that play an important role in machine learning models. In Bayesian inference, however, past information is evaluated in the same way as current information in the estimation process. As a consequence of this fact, in non-stationary environments where the state of the target changes during the estimation process, conventional Bayesian inference may not be so effective. For example, in the bandit problem, which is a kind of reinforcement learning problem, the Thompson sampling algorithm, a Bayesian inference-based policy, performs particularly well in steady-state environments, but performs extremely poorly in non-stationary environments. In a previous study, the extended Bayesian inference method was proposed that introduces forgetting and learning rate parameters into Bayesian inference. In this study, we use the extended Bayesian inference-based algorithm for the Bandit problem and conduct experiments in steady-state and non-stationary environments to validate the algorithm.