[2G5-OS-21e] 世界モデルと知能

[2G5-OS-21e-01] Integration of Large-scale Language Model and World Models in Robot Control

〇Akira Kinose¹, Ryo Okumura², Tadahiro Taniuchi^2,3 (1. Panasonic Connect Co., Ltd., 2. Panasonic Holdings Corporation, 3. Ritsumeikan University)

[[Online]]

Keywords:World Models, Robotics, Natural Language Processing

In general, robot controls are limited to pre-programmed actions, lacking adaptability to the environment and ease of use for humans. This paper proposes a method that combines a large-scale language model for code generation with a world model for robot control, allowing autonomous robot behavior from natural language sentences. The Language Model Program (LMP) generated by the large-scale language model uses the latent space of the world model learned by NewtonianVAE to enable high-freedom and high-abstraction control. This combination of large-scale language models and world models is important in the context of neuro-symbolic AI, which combines latent and symbolic representations.