[2M1-OS-19a] 世界モデルと知能(1/4)

[2M1-OS-19a-01] DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction

〇Masashi Okada¹, Tadahiro Taniguchi^2,1 (1. Panasonic Corp., 2. Ritsumeikan University)

[[Online]]

Keywords:World Model, Reinforcement Learning, Representation Learning

The present paper proposes a novel reinforcement learning method with world models, DreamingV2, a collaborative extension of DreamerV2 and Dreaming. DreamerV2 is a cutting-edge model-based reinforcement learning from pixels that uses discrete world models to represent latent states with categorical variables. Dreaming is also a form of reinforcement learning from pixels that attempts to avoid the autoencoding process in general world model training by involving a reconstruction-free contrastive learning objective. The proposed DreamingV2 is a novel approach of adopting both the discrete representation of DreamingV2 and the reconstruction-free objective of Dreaming. Compared to DreamerV2 and other recent model-based methods without reconstruction, DreamingV2 achieves the best scores on five simulated challenging 3D robot arm tasks. We believe that DreamingV2 will be a reliable solution for robot learning since its discrete representation is suitable to describe discontinuous environments, and the reconstruction-free fashion well manages complex vision observations.