[3M5-GS-10] AI application

[3M5-GS-10-04] A Presentation Training System Based on Multi-Modal Deep Neural Networks

Shengzhou Yi¹, Junichiro Matsugami ², Takuya Yamamoto³, Yukiyoshi Katsumizu³, 〇Toshihiko Yamasaki¹ (1. The University of Tokyo, 2. Rubato Co., Ltd., 3. P&I Information Engineering Co., Ltd.)

Keywords:Presentation, Assessment, Multi-modal

Presentation skills are one of the fundamental business skills for people today. However, learning these skills is difficult because people can acquire them only by their experiences. To solve this problem, we present a deep learning based system that can objectively evaluate both oral presentation and slide design and provide feedback for improvement to the users. For the speaking skill assessment, we train a multi-modal neural network including Bi-LSTMs and attention networks to analyze the linguistic and acoustic features of the oral presentation. The proposed network can predict the audiences' 14 types of impressions on the speakers' presentations with an average accuracy of 85.0\%. For the slide design analysis, we have realized a method that can visually analyze the slides independent of the file format and implemented it into the system. It can recognize whether the slide meet the requirement of ten assessment criteria. The average prediction accuracy of the proposed slide model is 81.67%.