[3Yin2] Interactive session 1

[3Yin2-30] Adversarial joint attack for walking robots

〇Takuto Otomo¹, Hiroshi Kera¹, Kazuhiko Kawamoto¹ (1.Chiba University)

Keywords:Reinforcement Learning, Robot control, adversarial attack

In the learning of walking robots by reinforcement learning, finding the environmental changes that reduce the reward leads not only to the detection of potential fall risks but also to the development of robust walking robots. In particular, the joints of a walking robot are prone to failures because they are movable parts in the environment, and environmental changes are likely to occur. In this study, we propose a method to find the joint torque limit that significantly reduces the reward by adversarial attack on the joint torque, even if the change is small. In general, adversarial attacks seek input perturbations that reduce the loss of deep learning, while the proposed method seeks environmental perturbations that reduce the reward of reinforcement learning. Therefore, efficient optimization algorithms such as the error backpropagation method cannot be used, and the differential evolution method is used to find the adversarial perturbation. In our experiments, we trained OpenAI Gym's Ant-v2 and Humanoid-v2 with deep reinforcement learning, and then searched for the torque limit that most hinders walking. With the proposed method, we were able to find a torque limit that significantly reduced the reward according to the gait of Ant-v2. On the other hand, Humanoid-v2 was found to be robust against perturbations to its joints.