[3Q1-OS-19a] Affective Computing

[3Q1-OS-19a-05] Analysis of Job Interview Training Feedback System Effectiveness Based on a Multimodal Machine Learning Model

〇Tomoya Ohba¹, Haruki Kuroki¹, Candy Olivia Mawalim¹, Shogo Okada¹ (1. Japan Advanced Institute of Science and Technology)

Keywords:Multimodal, Feedback System, Machine Learning, Job Interview

We built a humanoid agent system for VR experiences and collected a job interview data corpus.
The data corpus includes annotations of interview skill scores graded by third-party experts and self-efficacy annotations by the interviewees, for each question-answer.
The data corpus contains various kinds of multimodal data, including audio, biological (i.e., physiological), gaze, and language data.
In this study, we developed a feedback system for automated job interview training and analyzed the impact of the feedback.
The feedback system utilizes a machine learning model that uses acoustic and linguistic features.
In the control group, feedback was provided using a book.
The results of the comparison of the effects of the proposed system and the book suggested that the proposed feedback system could suppress the self-confidence of the group that tended to overestimate their performance when compared with the book.