[3Xin4] Poster session 1

[3Xin4-42] Target Object Identification in Mobile Manipulation Tasks Based on Sequence Labeling

〇Masaharu Morishita¹, Masatoshi Nagano¹, Tomoaki Nakamura¹ (1.The University of Electro-Communications)

Keywords:Natural Language Understanding, Mobile Manipulation, BERT

The mobile manipulation task, where a mobile robot grasps and moves a user-directed object, is one of the fundamental tasks of robots. In such a task, the user does not always explicitly mention the target object, and the robot has to understand the user's request from the abstract utterance. In this paper, we propose a method for selecting objects the user intends by using the information on objects in the environment as input, in addition to the abstract utterances of the user. In the proposed method, the target objects are selected by sequence labeling based on the BERT language model. This approach can generalize to objects that do not contain training data without retraining. In experiments, we created the training dataset and evaluated the performance of the proposed method by training the model on a dataset. As a result, we confirmed that the proposed method could identify the target object from objects and user utterances not included in the training data.