[1G5-OS-21b] 世界モデルと知能

[1G5-OS-21b-03] World models using latent diffusion model

Eitaro Yamatsuta¹, 〇Fumiya Uchiyama², Reiya Sekido³, Yuto Kawahara⁴, Masahiro Suzuki⁵, Yutaka Matsuo⁵ (1. Graduate School of Engineering, Osaka University, 2. The University of Tokyo, 3. Hokkaido University of Science, 4. Santa Moica College, 5. Graduate School of Engineering, the University of Tokyo)

Keywords:World models, Latent diffusion model, Rainforced Learning

World models model the external world from limited information and can be used to predict future external states and observations for learning. In spatio-temporal prediction, reinforcement learning methods using deep generative models have attracted attention. In generative models, Imagen and Stable-diffusion based on diffusion models are known for their high image generation capability. In this study, we propose a method to generate a better latent representation from the hidden states of LSTM by changing the vision part of World Models from conventional β-VAE to latent diffusion model, and compare these methods.