[2I5-GS-10] AI application:

[2I5-GS-10-02] Alpha-Q-Routing

Route Search Combining Partial Simulation and Estimated Time Tables

〇Akifumi Endo¹, Noboru Murata¹, Yusuke Maruyama², Felix Beierle² (1. Waseda University, 2. EAGLYS Inc.)

Keywords:routing, Reinforcement learning, OHT , optimization

Efficient route control in automated material handling systems, particularly Overhead Hoist Transport (OHT) systems, is a critical challenge in semiconductor manufacturing facilities. As production line conditions constantly change and unexpected congestion occurs, conventional shortest path algorithms based on mathematical optimization struggle to effectively avoid traffic congestion. To address this challenge, this research proposes a route control method called "Alpha-Q-Routing." Unlike conventional Q-learning-based methods that only utilized edge-level partial simulation results for Q-value updates, this method combines partial simulation results for multiple candidate routes with estimated travel times from Q-tables, enabling route selection based on more accurate arrival time predictions. Numerical experiments in an environment with 50 vehicles demonstrated that the proposed method achieved the highest number of completed deliveries compared to conventional methods, particularly proving its superiority in environments with high vehicle density. This achievement contributes to realizing efficient logistics management in dynamically changing manufacturing environments.