[3R1-OS-13b] OS-13

[3R1-OS-13b-01] A study of rapport estimation considering individual characteristics of speakers based on dyad interactions

〇Naoyuki SHINODA¹, Takato Hayashi¹, Ryo Ishii², Fumio Nihei², Atsuhi Fukayama², Shogo Okada¹ (1. JAPAN ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY, 2. NTT Corporation, Human Informatics Laboratories)

Keywords:multimodal signal processing, affective computing, rapport, bigfive

The long-term goal of this research is to realize optimal matching of dialogue partners, including the assignment of instructors for online lessons, by automatically estimating rapport, which represents "the state of connection and harmony between people" during dialogue. To achieve this goal, we propose a method that uses not only conventional multimodal features of speech, language, and vision, but also the BigFive scale of the speaker.
In this study, in addition to dialogue data based on several different themes with the same speaker combination, we use the results of a questionnaire about rapport toward the dialogue partner compiled after the dialogue, and the results of a questionnaire to measure the speaker's personal characteristics. The results of the experiment showed that the BigFive Scale was more accurate than the previous studies. As a result of the experiment, we were able to confirm that the accuracy of the BigFive scale improved in cases that included either or both sincerity and openness from those that did not. This experiment provides new insights into the influence of rapport and personal characteristics on interpersonal interaction.