[2Q4-OS-27b] 強化学習の新展開

[2Q4-OS-27b-03] Emulation in reinforcement learning

〇Reina Kitade¹, Takuma Wada², Yu Kono¹, Tatsuji Takahashi¹ (1. Tokyo Denki University, 2. Graduate School of Tokyo Denki University)

Keywords:Reinforcement learning, Bandit problems, Social learning

Just by referring to the outcome level of others, humans often achieve better performances through independent trial and error. This is a form of social learning called emulation. In emulation, often only a few bits of information, such as a new world record in an athletic event, leads to the performance improvement of the whole community. In a previous study, it was shown that the performance of all the individuals in a group can be improved more effectively by interpreting the others' results in a ``pessimistic'' way. For a single agent reinforcement learning, ``optimism in the face of uncertainty'' is well established as an effective principle. We suggested ``individual optimism and group pessimism under uncertainty'' for multiple agents. Pessimistic outcome level estimation methods such as the lower confidence bound (LCB) was shown to be effective in social bandit problems. However, LCB-based pessimism could not cope with more realistic non-stationary environments. In this study, we propose a new group-pessimism and its algorithm in multi-agent learning that can cope with various non-stationary environments.