[4N2-GS-7] Vision, speech media processing:

[4N2-GS-7-01] Enhancing Insights into Traffic Accident Risk with Multimodal Retrieval-Augmented Generation

〇Motoki Chiba¹, Osamu Ito¹, Takumi Iida² (1. Honda Motor Co., Ltd., 2. Elith Inc.)

Keywords:Multimodal, RAG, VLM

With the advancement of autonomous driving technologies, accurately estimating traffic accident risks has become increasingly important. Advanced Driver-Assistance Systems (ADAS), which use sensors for obstacle detection and provide collision mitigation and evasive steering, have reduced traffic accidents. However, the diversity and complexity of accident scenarios limit the potential of traditional ADAS technologies to achieve further reductions in accidents. Recently, Vision-Language Model (VLM) has been applied to the autonomous driving field. While VLMs possess broad knowledge and achieve reasonable accuracy in traffic scene understanding, they struggle to evaluate accident risks involving detailed and complex factors. Fine-tuning VLM specialized for traffic accident risk estimation is necessary, but the significant cost of data collection and annotation poses practical challenges. This study proposes a traffic accident risk explanation method using multimodal Retrieval-Augmented Generation (RAG) to improve explanation performance efficiently with minimal data. By leveraging a small amount of manually annotated data for retrieval and reference, the proposed method enhances explanatory capabilities for previously unseen images. Experimental results show that the proposed method improves traffic accident scene understanding compared to baseline model.