[4E2-OS-19a] OS-19 (1)

[4E2-OS-19a-02] Evaluating a Multimodal Meeting Summary Browser that Equipped an Important Utterance Detection Model based on Multimodal Information

〇Fumio Nihei¹, Yukiko Nakano¹ (1. Seikei University)

Keywords:Meeting summarization browser, Multimodal interaction, Important utterance estimation

This paper proposes a multimodal meeting summary browser with a CNN model that estimates important utterances based on co-occurrence of verbal and nonverbal behaviors in multi-party conversations. The proposed browser was designed to visualize important utterances and to make it easier to observe the nonverbal behaviors of the conversation participants. A user study was conducted to examine whether the proposed browser supports the user to correctly understand the content of the discussion. By comparing a text-based browser and a simple browser, it was found that the proposed browser was more efficient than the simple browser and allowed the user to obtain a more accurate understanding of the discussion than the text-based browser.