[4B3-GS-1] Fundamental AI, theory (2)

[4B3-GS-1-01] Reward distribution dependent adaptation of learning rate in mice exploration behavior

〇Hiroyuki Ohta¹, Kuniaki Satori², Yu Takarada², Masashi Arake¹, Yuji Morimoto¹, Ishizuka Toshiaki¹, Tatsuji Takahashi² (1. National Defense Medical College, 2. Tokyo Denki University)

Keywords:Reinforcement Learning, Exploration, Behavioral Science

This study examined how positive and negative learning rates for rewarded and unrewarded outcomes depend on reward distribution. The environment facing animals has huge number of choices, of which the rewarding options are sparsely distributed in space and time. Animals may change their behavior according to the frequency of rewards obtained from the environment. However, existing learning rate analyses focus on the response to a binary choice task, which does not allow the analysis of exploring multiple choices that are not frequently accessed. Thus, we performed 5-armed bandit task for mice and analyzed its exploration data by using Q-learning model with dual learning rates for positive and negative reward prediction errors. As a result, when the reward probability was small, it was estimated that the positive learning rate was about 6 times larger than the negative learning rate.