[1B3-OS-41a] OS-41

[1B3-OS-41a-02] Multimodal Prompt Analysis in Robotic Manipulation Tasks

〇Daiki Takahashi¹, Masahiro Suzuki², Yutaka Matsuo² (1. Aoyama Gakuin University, 2. Graduate School of Engineering, The University of Tokyo)

Keywords:AI, Robot, Multimodal

In this study, we analyzed multimodal prompts in a robot manipulation task, focusing on the interaction between textual and visual inputs. Using the VIMA benchmark, we evaluated the effects of modality dependence and the input order of observation tokens on the task success rate. The results revealed an overdependence on specific modalities and input order, indicating important issues in achieving robust multimodal learning. Our findings contribute to improve the generalizability of models in robot tasks.