[4F2-OS-25a] OS-25 (1)

[4F2-OS-25a-03] Multimodal Evoked Emotion Prediction and its Application to ASMR Video Analysis

Yu Yang¹, 〇Chie Hieida², Takato Horii^3,4, Takayuki Nagai^3,1 (1. The University of Electro-Communications, 2. Nara Institute of Science and Technology, 3. Osaka University, 4. International Research Center for Neurointelligence, The University of Tokyo)

Keywords:Evoked emotion prediction, Multimodal, Deeplearning, Autonomous sensory meridian response

With the development of digital terminals such as smartphones and tablets,videos that available for users to watch has reached to an enormous amount. In this context, applications such as classification, retrieval and distribution of personalized video content to meet the consumer needs remains a challenge. In general, humans tend to choose movies and music based on emotional characteristics. Therefore, evoked emotion analyzing may provide a guideline for this task. Emotions evoked by video are related to both audio and video modalities. In this study, we propose a deep learning model that estimates movie-evoked emotion by integrating multimodal information. Experiments using a movie database verify the change in estimation performance due to the integration of multimodal information, and show that the accuracy is improved compared to the conventional method. In addition, we analyze Autonomous Sensory Meridian Response (ASMR) videos, which have recently become a hot topic, and examine the relationship between evoked emotion and viewer behavior such as the number of views and likes/dislikes rates.