[2L1-J-9] Natural language processing, information retrieval: fusion with image

[2L1-J-9-01] Evaluation uncertainties in Image-Caption Retrieval

〇Kenta Hama¹, takashi matsubara¹, uehara kuniaki¹ (1. Kobe University)

Keywords:image caption retrieval, uncertainty, bayesian neural network

Deep learning algorithms are able to learn powerful representations for many tasks. These models’ outputs are often taken blindly and assumed to be accurate, however, which are not always the case. This blind assumption causes many issues such as AI unsafety and social bias. Therefore, a meaningful measure of uncertainty is essential. It has been shown that Monte Carlo (MC) Dropout can model epistemic uncertainty, and enhance model performance in machine learning tasks. In this paper, we propose an evaluation method of uncertainty in image caption retrieval and verified its significance by qualitative evaluation. Also, we show that a learning model using MC Dropout improves accuracy in image caption retrieval.