[2J5-GS-2] Machine learning: Advancement reinforcement learning (1)

[2J5-GS-2-05] Mastering a Game with Imperfect Information by Game Tree Search with a Latently Learned Model

In the Case of "Gyakuten Othellonia"

〇Shintaro Sakoda^1,2, Katsuki Ohto², Ikki Tanaka², Yu Kono² (1. Keio University, 2. DeNA Co., Ltd.)

Keywords:Deep Learning, Game Tree Search, Model-based Reinforcement Learning

In the field of board game AI, a technique that combines neural networks and tree search has attracted attention. In order to perform a tree search, the transition rules of the board need to be known. Researches on learning the transition rules of the state are also actively pursued as model-based reinforcement learning, and MuZero shows high performance in games such as Atari, Go, Shogi, and chess. In this study, we redefine MuZero's algorithm as supervised learning and examine a method to apply it to the more complicated game "Gyakuten Othellonia". When the MuZero algorithm was applied directly to "Gyakuten Othellonia", the performance is partially improved, but it is shown that errors in transition prediction could adversely affect the tree search. The analysis suggests that a tree search to deal with uncertainty could improve performance further.