[2Win5] Poster session 2

[2Win5-77] A Preliminary Study on UI Element Detection for Mobile Games Towards Automated Testing

〇Nozomu Karai¹, Koya Ihara¹, Tatsuya Ishimoto², Daiki Kubo³, Kotaro Kikuchi¹ (1.CyberAgent, Inc., 2.TOKYO GameFi, Inc., 3.Applibot, Inc.)

Keywords:Mobile Games, UI Element Detection, Automated Testing

Developing attractive mobile games requires extensive testing to ensure the quality and reliability of their rich graphics and user interface (UI). Frequent UI changes during development necessitate vision-based testing; however, detecting UI elements in mobile games has not been fully explored compared to web pages and mobile apps. In this study, we benchmark recent UI detection methods using our new dataset constructed from three different types of mobile games. We compare the methods with several evaluation metrics, including three proposed metrics designed for automated testing. The experimental results show that existing methods still face difficulty in accurately detecting UI elements, yet they could be practically used for specific purposes such as monkey testing.