[2O1-GS-7] Vision, speech media processing: generation

[2O1-GS-7-01] Generating Subgoals with Adversarial Network on Vision-and-Language Navigation

〇Shintaro Ishikawa¹, Komei Sugiura¹ (1. Keio University)

Keywords:Vision-and-Language Navigation, Adversarial Training, Robot, Natural Language Processing, Image Processing

In this paper, we focus on a vision-and-language task in which a robot is instructed to execute household tasks. We propose Moment-based Adversarial Training (MAT), which uses two types of moments for perturbation updates in adversarial training. We introduce MAT to the embedding spaces of the instruction, subgoals, and state representations to handle their varieties. We validated our method on the ALFRED benchmark, and the results demonstrated that our method outperformed the baseline method for all the metrics on the benchmark.