[1J4-OS-10b] OS-10

[1J4-OS-10b-03] Pseudo Training Data Generation for Automatic Aggregation of Open-Ended Questionnaire Responses by Large Language Models

〇Ryo Hasegawa¹, Yuki Zenimoto¹, Takehito Utsuro¹, Hiromitsu Nishizaki², Masaharu Yoshioka³, Noriko Kando⁴ (1. University of Tsukuba, 2. University of Yamanashi, 3. Hokkaido University, 4. National Institute of Informatics)

Keywords:large language models, open-ended questionnaire, pseudo data, automatic aggregation

Analyzing surveys utilizing open-ended responses to questionnaires is a
valuable approach to elucidating respondents' perspectives and opinions,
thereby gaining insights. However, the analysis of responses on a large
scale necessitates a considerable amount of manual labor. Thus, this
paper takes an approach of automating the analysis of open-ended
responses using large language models. We have generated several types
of pseudo data for training category classification models and evaluated
the performance of the models trained on each dataset. Through this
process, we examine the performance improvements of category
classification models using the pseudo datasets automatically generated
and annotated by large language models. Evaluation results show that,
through several operations of pseudo open-ended responses, we improved
the category classification performance against real open-ended
responses from 77% to 83%.