[4Xin2] Poster session 2

[4Xin2-100] Customer Service Evaluation using Roundtable with Multiple Large Language Models

〇So Watanabe¹, Chee Siang Leow¹, Hiromitsu Nishizaki¹, Junichi Hoshino², Takehito Utsuro² (1.University of Yamanashi, 2.University of Tsukuba)

Keywords:Large Language Models, Customer Service Training

This paper proposes a method for correcting store staff's customer service speech using multiple large language models (LLMs) to improve the quality of the speech of store staff in customer service. In addition, the staff's speech is scored and a commentary is generated to provide a basis for scoring. By correcting the staff's utterances and providing a quantitative evaluation, appropriate feedback can be provided to the staff. This study adopts a roundtable method called ReConcile as one of the methods to utilize multiple LLMs. The results of evaluation experiments showed that by refining the output of multiple LLMs with ReConcile, it was possible to modify the utterances more appropriately than a single LLM and to assign evaluation points that were closer to human senses.