[3G5-OS-24b] 日常生活知識とAI

[3G5-OS-24b-02] Automatic Generation of Action Scripts from Descriptions of Daily Life Activities in VirtualHome

〇Jin Aoyama¹, Motoki Yatsu¹, Takeshi Morita¹ (1. Aoyama gakuin University)

Keywords:VirtualHome, Action Script, Descriptions of Daily Life Activities

Agents in the household simulator named VirtualHome can interact with the environment based on action scripts that describe a sequence of instructions. Each instruction consists of an action and an object used by the action. There is a dataset containing activity labels, activity descriptions, and action scripts that correspond to each activity description built through crowdsourcing. However, the construction cost is high, and there is an issue that action scripts that cannot be executed in VirtualHome are included in the dataset. Since actions that can be performed on VirtualHome have preconditions for execution, the list of actions generated by existing methods often cannot be executed on VirtualHome. To solve the issues, we propose a method to generate action scripts from descriptions of daily life activities by natural language processing technologies and home environment knowledge. In addition, we evaluated the proposed method using the above dataset.