[2G1-OS-21c] 世界モデルと知能

[2G1-OS-21c-04] A Transfer Learning Method for the Multimodal Language Understanding Based on Dual ProtoNCE

〇Seitaro Otsuki¹, Shintaro Ishikawa¹, Komei Sugiura¹ (1. Keio University)

Keywords:Dual ProtoNCE, Transfer Learning, Multimodal Language Understanding

We focus on the task of identifying target objects in domestic environments according to free-form natural language instructions. In this study, we propose a novel transfer learning approach for multimodal language understanding, Prototypical Contrastive Transfer Learning (PCTL) which uses a new contrastive loss, Dual ProtoNCE. Our experiment demonstrated that PCTL outperformed existing methods.