[1D5-GS-10] AI application: Movement

[1D5-GS-10-03] Explanation of Traffic Risks with LLM Using GIS Data and Street Images

〇Ryota Mimura¹, Kota Shimomura^2,3, Atsuya Ishikawa¹, Osamu Ito¹, Kazuaki Ohmori², Ryuta Shimogauchi², Reoto Wakabayashi², Koki Inoue² (1. Honda R&D Co., Ltd., 2. Elith Inc., 3. Chubu University)

Keywords:Geographic Information System, Geospatial Data, Large Language Model

Consideration of traffic risk in driver assistance systems and automated driving technology is important in preventing traffic accidents. Traffic risks are considered to be contained in image information. However, it is difficult to explain traffic risk in driving scenes from image information alone, and research in this area has not yet progressed sufficiently. In this study, we propose a multimodal framework that can explain traffic risks by using GIS data and street images. This framework identifies the coordinates of high-risk areas from traffic accident risk maps created based on GIS data and trains a multimodal network using street images associated with those areas. By doing so, we construct a framework that effectively explains traffic risk in an arbitrary scene. Experimental results show that the proposed framework can generate captions that explain traffic risks for high-risk areas based on GIS data.