[1H4-GS-1c] 基礎・理論：アルゴリズム

[1H4-GS-1c-02] Transfer in reinforcement learning with indeterminate natural transformations

〇Takuma Wada¹, Tatsuji Takahashi^1,2 (1. Tokyo Denki University, 2. RIKEN AIP)

Keywords:Reinforcement Learning, Transfer Learning, Category theory, Analogy Inference, MDP Homomorphism

Even when solving unknown tasks, humans are able to efficiently solve a wide variety of tasks in front of them by utilizing their knowledge and past experience they have gained in other domains. On the other hand, agents in Reinforcement Learning, which learns strategies based on rewards in an unknown environment, requires a lot of trial and error because it does not have knowledge of other environment and therefore cannot efficiently search using its experience. The solution to this problem is transfer learning, which is the adaptation of knowledge learned in another domain to a new domain. In this study, we focus on the cognitive function of analogy as a form of transfer. One model of analogy is the theory of indeterminate natural transformation (TINT) proposed by Fuyama and Saigo. It is an algorithm that constructs an appropriate functor by searching for natural transformations that displace the obvious functors in category theory. By using TINT in reinforcement learning, we aim to find a correspondence (functor) between the experience learned in another task and the experience in a new task.