[1J2-GS-10d] AI応用：分類と論理

[1J2-GS-10d-03] A study of multimodalization of spatio-temporal neural network for prediction

〇Yutaro Mishima^1,2, Shinya Wada¹ (1. KDDI Research, Inc., 2. KDDI Corporation)

Keywords:Spatio-Temporal Data, Multimodal, Deep Learning, Air quality, Forecasting

These days, many novel neural networks for modeling spatio-temporal relationship are proposed as many kinds of spatio-temporal datasets like location dataset or traffic dataset are published and utilized. However, novel networks have a common problem that they cannot handle properly multimodal data with complex (multi-step) relationship, e.g. Modal A affects modal B and modal B affects modal C. This problem must be solved because much more kinds of spatio-temporal data will be distributed in the future. In this paper, for discussing what kind of structure “multimodal spatio-temporal network” should be, we conduct some preliminary experiments which includes extending existing spatio-temporal network to handle multimodal data and comparing prediction capability with the original network. Based on the results, we conclude that “multimodal spatio-temporal network” should properly encode the information which affects relationship of modals dynamically, e.g. meteorological data.