JSAI2018

Presentation information

Oral presentation

General Session » [General Session] 11. Robot / Real World

[2A3] [General Session] 11. Robot / Real World

Wed. Jun 6, 2018 3:20 PM - 5:00 PM Room A (4F Emerald Hall)

座長:稲邑 哲也(国立情報学研究所)

4:20 PM - 4:40 PM

[2A3-04] Understanding Ambiguous Instructions Using Generative Adversarial Nets for Object Disposal Tasks

〇Komei Sugiura1, Aly Magassouba1, Hisashi Kawai1 (1. NICT)

Keywords:Deep Neural Network, multimodal language understanding, Generative Adversarial Nets, Domestic Service Robots, Ambiguity

This paper focuses on a multimodal language understanding method for ``Carry and Place'' tasks with domestic service robots. We address the case of ambiguous instructions, that is when the target area is not specified. For instance ``Put away the milk and cereal.'' is a natural instruction where there is ambiguity on the target area, considering daily life environments. Conventionally, this instruction can be disambiguated from a dialogue system, but at the cost of time and cumbersomeness. Instead, we propose a multimodal approach, where the instructions are disambiguated from the robot state and environment context. We develop MultiModal Classifier Generative Adversarial Network (MMC-GAN) to predict the likelihood of the different target areas considering the robot physical limitation and the target clutter. Our approach, MMC-GAN, significantly improves accuracy compared to baseline methods using instructions only or simple deep neural networks.