[3H3-OS-12a] グループインタラクションとAI(1/2)

[3H3-OS-12a-05] Explainable Models for Predicting Interlocuters’ Subjective Impressions based on Nonverbal Functional Features

When and What Kind of Behaviors Affected Interlocuters’ Impressions?

〇Shumpei Otsuchi¹, Kazuki Miyoshi¹, Yoko Ishii², Ryo Ishii², Shin-ichiro Eitoku², Kazuhiro Otsuka¹ (1. Yokohama National University, 2. Nippon Telegraph and Telephone Corporation)

Keywords:Nonverbal Behavior, Social Signal Processing, Multiparty Conversation, Deep Learning, Explainable AI

An explainable framework is proposed to predict the interlocutors' subjective impressions in group meetings. The goal is to explain when and what kind of nonverbal behaviors affected interlocutors' impressions during meetings. To that end, we formulate a two-fold framework consisting of the regression models of interlocutors' impression scores based on functional head-movement features, followed by the estimation of the temporal distribution of SHAP-based feature contribution, which is obtained with the kernel density estimation of the temporal occurrence probabilities of head-movement functions. The former stage identifies the behaviors related to the impressions, and the later stage suggests the timing of the behaviors, by locating the maximum point of the temporal feature contribution curve, based on an assumption that temporary intensive behaviors lead to form a strong impression. This report shows preliminary results and analyses applied to 4-party 17-group discussions.