[4Xin1] Poster session 2

[4Xin1-68] Development of a semi-autonomous teleoperation system for a CG avatar using a deep generative model

Yusuke Nishimura^1,2, Shota Takashiro², Yuya Okadome¹, Hiroshi Ishiguro², 〇Yutaka Nakamura¹ (1.Riken R-IH GRP, 2.Osaka University)

Keywords:Communicative intelligence, Human-machine symbiotic society, Deep generative model, Neural Network, CG avatar

In avatar teleoperation, it is necessary to recognize the situational cues of the interlocutor, such as their behaviors, in order to operate appropriately. However, precise control is not easy due to limitations of the sensor and communication systems, as well as the complexity of the operation. Therefore, the development of a teleoperation system that is easy for operators to use and provides rich expression for users is required. In this study, we propose a semi-autonomous teleoperation system that automatically controls some modalities using a deep generative model of human behaviors in dialogues. The experimental results, applied to a video dialogue system using a CG avatar, suggested that it is possible to generate behaviors recognized as attentive listening through teleoperation.