[2J1-GS-8a] ロボットと実世界：人との協同

[2J1-GS-8a-01] Real-time Imitations of Human Motion for Robots using GPT-2 and VQ-VAE

〇Kouichirou Kawasaki¹, Kazuki Miyazawa¹, Takato Horii¹, Takayuki Nagai^1,2 (1. Osaka University, 2. The University of Electro-Communications)

Keywords:Imitation Movement, Motion Prediction, GPT-2

In this paper, we propose a system in which a robot can imitate human motion in real time by estimating human joint angles from camera images and predicting subsequent motions.To estimate the joint angles, VIBE that can predict joint positions in three dimensions using a single camera is involved. Future motion prediction is performed by discretizing the motion information using VQ-VAE and predicting the discretized motion information using GPT-2. By compensating the time consumed for information processing and data transmission, the real-time imitation of human motion by robots can be achieved. In the preliminary experiment using three types of human movements, it is revealed that the robot is able to follow and imitate the real human motion in real time using the proposed method.