[4G1-GS-6] Language media processing:

[4G1-GS-6-01] Development of a Fine-tuned Local Generative AI-based Support Tool for Scoring English Performance Tests

〇Jun Kaneko¹, Takashi Otsuki², Takayuki Sakaguchi², Jesse Sokolovsky¹, Hiroki Shoyama³, Dai Inoue⁴ (1. Mie University, 2. Yamagata University, 3. Junior High School attached to the Faculty of Education, Mie University, 4. Elementary School attached to the Faculty of Education, Mie University)

Keywords:Generative AI, Fine-tuning, Education Aid

In educational settings, performance tests are administered to assess students’ English speaking and writing skills. However, grading these assessments can be a significant burden for school teachers. To alleviate this, automated scoring using AI is being explored. Many existing approaches calculate, aggregate, and evaluate scores for each assessment criterion. Nevertheless, determining what criteria to use, how many to include, and how to weigh them, as well as establishing evaluation standards, can be challenging and time-consuming. Furthermore, these services are often provided by commercial businesses, raising concerns about costs and personal data protection. The current project represents an attempt to address these issues by fine-tuning an open-source generative AI model in a local environment to create a customized AI for evaluating English performance tests. The three perspectives outlined by the Ministry of Education, Culture, Sports, Science and Technology—“Knowledge and skills,” “Abilities to think, make decisions and express oneself,” and “Student attitudes about proactive learning”—were used to label the training data, and the model was fine-tuned accordingly. The results suggest that it is feasible to develop a system that supports performance test evaluation.