[1N4-GS-10] AI application

[1N4-GS-10-04] Automated Short Answer Scoring using a Small Amount of Training Data

〇Ryo Haga¹, Hidekatsu Ito¹, Masaki Ishii¹, Kohji Dohsaka¹ (1. Akita Prefectural University)

Keywords:natural language processing, automatic scoring, educational support

Automated short answer scoring is the task of estimating the score of a short text answer written in response to a given prompt, based on pre-established rubrics. This research aims to automatically score written Japanese answers, utilizing a limited amount of training data while maintaining the scoring accuracy of previous research. We consider a scenario in which a mere 50 human-scored answers and 200 unscored answers are available as a limited amount of training data. Our proposed method fine-tunes the GPT-2 model with 200 unscored answers, generates answers using the fine-tuned model, and scores them using an analytic score prediction model trained on human-scored answers. A scoring model is then constructed using the scored generated answers. Our experimental results demonstrated that the scoring model could not attain the same level of scoring accuracy as previous research using 50 human-scored answers and 200 unscored answers. Nevertheless, we discovered that the model's performance improved as the number of scored generated answers increased. Unlike previous research that trained the scoring model using holistic scores and annotations in addition to analytic scores, we used only analytic scores. This indicates that the scoring accuracy of previous research can be achieved using a small amount of training data, by employing annotations and holistic scores.