[1Q4-GS-11] Robot and real worlds: Machine learning

[1Q4-GS-11-02] Learning Alignment Tasks Based on Residual Reinforcement Learning with Multiple Expert Policies

〇Kazuki Yaginuma¹, Tomoaki Nakamura¹, Yusuke Kato^2,3, Takayuki Nagai^1,4, Jun Ozawa^2,3 (1. The University of Electro-Communications, 2. Advanced Industrial Science and Technology, 3. Panasonic Corporation, 4. Osaka University)

Keywords:Residual Reinforcement Learning, Reinforcement Learning, Gaussian process-hidden semi-Markov model (GP-HSMM)

Reinforcement learning (RL) enables robots to flexibly learn skills from the interaction with the environment.

However, robots move randomly to explore valuable actions at an early stage, which can be unsafe.

Moreover, it takes a long time to learn motion from scratch.

To learn more efficiently and safely, residual reinforcement learning (RRL) has been proposed.

In RRL, skills are learned by correcting expert policy that can be obtained from an expert's demonstration.

However, conventional RRL assumes a single expert policy, whereas we consider multiple policies for more complex tasks.

In this paper, we propose RRL with multiple expert policies, where a selection of a suitable expert policy in the current state is also learned based on RL.

Experimentally, we show that the agent can learn more accurate skills in the object alignment task.