[4G4-GS-2m] 機械学習：学習方略(2/2)

[4G4-GS-2m-01] Learning a Hierarchical Recurrent State Space Model in Complicated Environments

〇Keno Harada¹, Masahiro Suzuki¹, Yutaka Matsuo¹ (1. the University of Tokyo)

Keywords:Deep Learning, Temporal Abstraction, State Space Model

Temporal abstraction is considered to contribute sample efficiency in model-based reinforcement learning. The previsously proposed models for temporal abstraction has been experimented in simple environments. However, for learning behavior policy in real world such as home service robots, it is necessary to test if temporal abstraction can be accomplished in complicated environments where high-resolution observations can be obtained and where objects composed of multiple colors and non-plain patterns exist, rather than an existing experimental environment where only simple and low-resolution observations can be obtained. We believe that the abstraction of observations in a complex environment requires the use of encoders to extract useful information. We train a hierarchical recurrent state-space model, which is one of the models for temporal abstraction, on a complex environmental data set and show that VAE pretraining technique for encoder improves the performance of the model in abstracting observation states and predicting future transitions given contextual data, compared to the case where the model is trained without the pretraining technique through evaluation experiments.