[2K6-ES-2] Machine learning: Modeling

[2K6-ES-2-04] Balancing policy improvement and evaluation in Risk-sensitive Satisficing algorithm

〇Hiroaki Wakabayashi¹, Takumi Kamiya¹, Tatsuji Takahashi¹ (1. Tokyo Denki University)

Keywords:Satisficing, Reinforcement Learning

Reinforcement learning is a type of machine learning in which an agent learns an appropriate action sequence through purely trial-and-error type interaction with the environment. In contrast, human beings who live in a complicated environment often make decisions based on rules called satisficing which is different from optimization: we confront a task with a certain reference level (called aspiration level) and cease searching when we have found a satisfactory action above the reference level. A simple value function that is called Risk-sensitive Satisficing (RS) was proposed and proven its optimal exploration behavior. Furthermore, the global reference conversion GRC algorithm allows us to assign the aspiration value to each state allocated from the global aspiration. However, there are some issues left in the current GRC. In this study, we clarify the problems of GRC using Suboptima World and verified whether one of the problems could be solved by using eligibility trace. As a result of the verification, it is shown that the RS using the eligibility trace was able to correctly evaluate the degree of satisficing.