[2C4-GS-2] Machine learning: reinforcement learning (1)

[2C4-GS-2-03] Natural reinforcement learning that takes into account the degree of achievement of the aspiration level

〇Shunpei Koshikawa¹, Shuichi Arimura², Hiroaki Wakabayashi², Yu Kono¹, Tatuji Takahashi¹ (1. Tokyo Denki University, School of Science and Engineering, 2. Graduate School of Tokyo Denki University)

Keywords:Machine Learning, Reinforcement Learning, Satisficing

Imitating intelligence that is as flexible and complex as that of humans is still a far-reaching goal. But a technique called deep reinforcement learning (RL) , which allows machines to autonomously obtain action procedures, come to show score as high as humans in some kind of games. Deep learning (DL) in general, however, is a learning method that requires a lot of data, meaning that it does not in the first place go well with RL, that learns from actively collected data in some particular and possibly limited opportunities. On the other hand, in many cases the actual desire of the agent is to obtain an action procedure that is above a certain level. This "certain level" means, for example, the living cost in physiology. And humans learn various things drawing on some general level of achievement that the society forces to comply. To model those kinds of learning that aims to achieve some natural level of aspiration, a method called natural RL has been proposed. This method aims to speedily achieve certain aspiration levels, thus enabling at least to be compatible with DL. Though current RL algorithm reflects degree of aspiration level of whole task in each state, it has a problem in that the aspiration level cannot be estimated accurately. In this paper, we consider natural RL technique that doesn’t depend on advanced parameter design, through analysis difference between estimate policy and behaviour policy.