[3Rin4-52] Investigation into Annotation Aggregation Criterion for Multi-label Emotion Classification
Keywords:emotion analysis, multi-label classification, annotation aggregation
In narrative understanding, it is crucial to accurately estimate and track the implicit emotions of characters.
This paper investigates the impact of a method for determining the ground-truth emotion labels on emotion estimation accuracy. In the method, a total score is first calculated by aggregating raw label scores assigned by annotators, and then the ground-truth labels are selected by applying a threshold. We experimentally investigated the changes in estimation accuracy when the score aggregation method and the threshold were altered in the learning and testing, respectively. As a result, higher accuracies were observed compared to the usual setting, where the same criterion is used both in training and testing, when using the training data created with the criterion looser than that of the testing. This result suggests that it may be beneficial to train a classifier by using the dataset created based on looser criterion than the expected operational criterion.
This paper investigates the impact of a method for determining the ground-truth emotion labels on emotion estimation accuracy. In the method, a total score is first calculated by aggregating raw label scores assigned by annotators, and then the ground-truth labels are selected by applying a threshold. We experimentally investigated the changes in estimation accuracy when the score aggregation method and the threshold were altered in the learning and testing, respectively. As a result, higher accuracies were observed compared to the usual setting, where the same criterion is used both in training and testing, when using the training data created with the criterion looser than that of the testing. This result suggests that it may be beneficial to train a classifier by using the dataset created based on looser criterion than the expected operational criterion.
Authentication for paper PDF access
A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.