[2Win5] Poster session 2

[2Win5-14] Enhancing Exploration Efficiency in Sparse Reward Environments through Refined Intrinsic Motivation and Contrastive Learning

〇Hiroki Sagara¹, Shoma Yato², Shuichi Kusaba³ (1.Kyushu University, 2.Tokyo Polytechnic University, 3.University of Tokyo)

Keywords:Intrinsic Motivation, Contrastive Learning

In this study, we propose an improved method for intrinsic motivation and contrastive learning to enhance exploration efficiency in reinforcement learning environments with sparse rewards. Intrinsic motivation guides agents toward novel states by providing internal rewards based on prediction errors. However, existing methods, such as Self-supervised Network Distillation (SND), are vulnerable to noise introduced by the agent’s noisy actions. To address this issue, we introduce an enhancement that strengthens contrastive learning by treating temporally adjacent frames as positive examples. This modification improves novelty detection and enables sustained exploration. Experimental results in sparse-reward environments, including the Procgen benchmark, demonstrate improvements, with our method achieving external rewards in half the training steps compared to baseline models.