[2M5-OS-19c] 世界モデルと知能(3/4)

[2M5-OS-19c-04] Effect of Temporal Discretization in Control Policy Learning Algorithms and Their Extensions to Continuous Time

〇Tatsuya Matsushima¹, Jumpei Arima², Kaito Suzuki³, Yusuke Iwasawa¹, Yutaka Matsuo¹ (1. The University of Tokyo, 2. Matsuo Institute, 3. Tohoku University)

Keywords:Robot Learning, Reinforcement Learning, Deep Reinforcement Learning, Temporal Discretization

There have been many attempts, mainly in deep reinforcement learning domain, to learn end-to-end control policies using high-dimensional inputs like images, and their effectiveness has been verified mainly with video games and simple robot simulators.
Many of the environments used in these studies are designed with the assumption that time is discretized and that observations and actions operate synchronously.
This is different from the nature of real robotic systems, where various sensors and actuators operate asynchronously at different frequencies and computation time needs to be taken into account, which can be the reasons why learning end-to-end policies is difficult in the real world.
After summarizing and discussing previous works on time discretization, this paper presents experiments using algorithms and robot simulation environments frequently used in deep reinforcement learning to verify and discuss the effect of time discretization on the performance of policy learning.