[2U1-IS-1b] Knowledge engineering

[2U1-IS-1b-02] Towards Commonsense Reasoning in Outdoor Visual Linguistic Navigation

〇Anirudh Reddy Kondapally^1,2, Kentaro Yamada², Hitomi Yanaka¹ (1. The University of Tokyo, 2. Honda R&D Co.,Ltd., Tokyo, Japan)

[[Online, Regular]]

Keywords:Commonsense, AI reasoning, Visual Linguistic Navigation

The advent of deep learning models has made considerable strides in tasks related to navigation in the real world such as object detection and path planning. It has also led to the development of a more complicated task of visual-linguistic navigation (VLN) i.e., dialogue for navigation. Among VLN variations, outdoor scenes are significantly more difficult than indoor because of the randomness inherent in an uncontrolled environment. Outdoor VLN is also said to be closer to the reasoning required in the real world. However, the datasets available for Outdoor VLN tasks have been focused mainly on judging spatial reasoning abilities. This is not enough to create systems that work in real life as there is a need for commonsense reasoning abilities i.e. social and event-based reasoning. We create a small benchmark commonsense reasoning-based dataset and evaluate the performance of state-of-the-art VLN models on it. From our findings, we show that there is a need for commonsense reasoning-based datasets.