[2S4-GS-2] Machine learning:

[2S4-GS-2-04] Online Decision Transformer under Frame Dropping

〇Daiki Yotsufuji¹, Kenta Nishihara¹, Shoma Shimizu¹, Kento Uchida¹, Shinichi Shirakawa¹ (1. Yokohama National University)

Keywords:Frame Dropping, Online Reinforcement Learning, Decision Transformer

In real-world reinforcement learning applications, communication delays or sensor failures often cause frame dropping, in which the agent cannot receive dropped states and rewards. Because frame dropping degrades the agent's performance, the decision transformer under random frame dropping (DeFog) was developed by incorporating additional mechanisms into the decision transformer to tackle frame dropping. Although DeFog can mitigate performance degradation in frame-dropping environments, since DeFog is an offline learning method, it is difficult to select appropriate actions for the states not included in the dataset. In this study, we propose OnDeFog, which integrates the mechanisms in DeFog with the online decision transformer (ODT), an online reinforcement learning method that learns policy through direct interaction with the environment. The experimental results show that OnDeFog outperforms ODT in environments with a high dropping frame rate and is superior to DeFog with datasets containing a large amount of low-reward data.