[3Rin4] Interactive 1

[3Rin4-52] Investigation into Annotation Aggregation Criterion for Multi-label Emotion Classification

〇Hikari Tanabe¹, Tetsuji Ogawa¹, Tetsunori Kobayashi¹, Yoshihiko Hayashi¹ (1.Waseda University)

Keywords:emotion analysis, multi-label classification, annotation aggregation

In narrative understanding, it is crucial to accurately estimate and track the implicit emotions of characters.
This paper investigates the impact of a method for determining the ground-truth emotion labels on emotion estimation accuracy. In the method, a total score is first calculated by aggregating raw label scores assigned by annotators, and then the ground-truth labels are selected by applying a threshold. We experimentally investigated the changes in estimation accuracy when the score aggregation method and the threshold were altered in the learning and testing, respectively. As a result, higher accuracies were observed compared to the usual setting, where the same criterion is used both in training and testing, when using the training data created with the criterion looser than that of the testing. This result suggests that it may be beneficial to train a classifier by using the dataset created based on looser criterion than the expected operational criterion.