[3C4-GS-6] Language media processing

[3C4-GS-6-05] Instructional Utterance Generation Using Linguistic Representations of Situations in Collabrative Environment

〇Riku Miyashita¹, Akinobu Lee¹ (1. Nagoya Institute of Technology)

Keywords:Minecraft Dialogue Corpus, Collaborative Environment, Instructional utterance generation

The Minecraft Dialogue Corpus is a dataset that contains conversations and action histories of an Architect, who has a blueprint, and a Builder, who follows the instructions and places blocks, as they work together to build a 3D structure in a virtual environment. In this study, we specifically deal with a task in which the system acts as the Architect side and gives instructions to a human Builder. In this task, it is necessary to build an Architect model that can generate appropriate instructional utterances according to the environmental situation. In this paper, we propose an instructional generation system based on a GPT2 that uses a simple text representing the environmental situation and the speech history as input. The results of the experiment showed that the accuracy of the color of the blocks to be placed improved and tended to generate longer and more diverse response sentences.