[1N3-IS-5b] Human interface, education aid (2/2)

[1N3-IS-5b-03] A Way-of-Talking Style Modification Assistance System Based on Apparent Personality

〇Nathan Boyer^1,2, Mitsuhiko Kimoto¹, Kohei Okuoka¹, Yuki Abe¹, Michita Imai¹ (1. Keio University, 2. École Centrale de Lille)

Keywords:Apparent personality, Communication aid, Prediction explanation, Regression

Apparent personality (AP) impacts first impressions; people want to know how they are perceived by others, and how to change the image they give. Related research only focus on predicting accurately AP, without exploring how to change it. We propose a system that predicts its user’s AP traits according to the Big Five model, and gives them feedback on how to modify their way-of-talking style in order to reflect a desired AP. We extract 11 audio features from recorded speech, that can be interpreted simply (e.g. in terms of speed, loudness...). These features are then fed to a LightGBM regressor, which predicts an AP by giving a score in each Big Five model’s category. Finally, we determine each feature’s impact on the prediction by computing its Shap value, and give meaningful advice on what to do to change each category’s score. Our prediction system achieves a mean accuracy of 89.25% on all five categories, using only a few features and a simple regressor, which is competitive with current state-of-the-art systems that use deep neural networks and millions of features. Furthermore, we propose a first simple system that gives feedback using the features’ Shap values.