[2B5-GS-2] Machine learning: Reinforcement learning

[2B5-GS-2-02] Goal-specific state space reduction in incomplete information games

〇Kazuki Takahashi Takahashi¹, Tomoki Fukai², Yutaka Sakai³, Takashi Takahashi Takekawa¹ (1. Kogakuin University, 2. Okinawa Institute of Science and Technology, 3. Tamagawa University)

Keywords:Reinforcement learning, Bayesian inference, State reduction

In incomplete information games, it is difficult to predict the opponent's strategy, there has been a lot of research on finding a Nash equilibrium, which is a strategy that is easy to win independent of the opponent's strategy. Poker, which has a huge observable value space of 10¹⁶, uses Deep Neural Networks (DNNs) to find Nash equilibrium strategies and has achieved performance superior to that of humans. On the other hand, it is difficult to explain the appropriateness of the selected action in terms of the complex state space. In this study, we propose a Bayesian model that reduces a huge observation space to a concise state space and evaluates its performance using the incomplete information game "Vulture Culture" as a subject. As a result, the proposed method reduces an observation space of about 10⁴ to a near-optimal state space. It is also shown that the appropriate state space reduction facilitates the prediction of the opponent's strategy and improves the learning speed of the optimal strategy.