[2Win5] Poster session 2

[2Win5-83] A note on motion recognition for tire inspection based on the cooperative use of object tracking models and video-LLMs

〇Kyohei Kamikawa¹, Ren Togo¹, Keisuke Maeda¹, Takahiro Ogawa¹, Miki Haseyama¹ (1.Univ. of Hokkaido)

Keywords:Industrial applications, Object tracking, Large language model (LLM), Video-LLM, Dense Video captioning

There are multiple inspection steps for each part of the tire, and it is important to implement these steps accurately. Therefore, introducing action recognition technology is needed for tire inspection. In this paper, we propose a method for action recognition in tire inspection based on the cooperative use of the object tracking model and Video-LLM to support tire inspection. Dense video captioning, which provides detailed captions for each video segment, has been developed with the advent of Video-LLM. Since focusing on detailed actions in the video and performing specialized recognition has been difficult, our method enables Video-LLM to capture detailed captions for specific objects by cropping objects that are important for inspection action recognition from video based on an object tracking model. We quantitatively demonstrate the effectiveness of this method through experiments using actual tire inspection videos.