[2L5-GS-1] Fundamental AI, theory, algorithm:

[2L5-GS-1-03] A Generative Model for Diverse Policies Based on Post-hoc Inference of Latent Intentions

〇Yu Kono^1,2 (1. DeNA Co., Ltd., 2. Tokyo Denki University)

Keywords:Generative Model, Reinforcement Learning, Machine Learning

Recent advances in large-scale generative models have enabled the recognition, generation, and transformation of various modalities, such as language, speech, images, and video. While this suggests significant progress toward general AI, intelligent agents mainly aim to adapt to environments by generating actions. However, generating action concepts and policies remains underdeveloped, with only limited success through complex prompt engineering and no solid theoretical foundation. Reinforcement learning (RL) is a primary method for generating actions, but it requires extensive training to find a single optimal policy, leading to poor cost-effectiveness and a lack of diversity in the generated policies. Despite this, RL maintains strong exploratory abilities. This study proposes a hybrid model that combines the diversity of generative models with the exploration ability of RL. Inspired by the human tendency to retroactively assign meaning to actions, we introduce latent variables into the policy parameters and embed trajectories using a specialized generative model. Learning is successfully achieved through methods like contrastive learning and stationary distribution estimation.