JSAI2024

Presentation information

Poster Session

Poster session » Poster session

[4Xin2] Poster session 2

Fri. May 31, 2024 12:00 PM - 1:40 PM Room X (Event hall 1)

[4Xin2-100] Customer Service Evaluation using Roundtable with Multiple Large Language Models

〇So Watanabe1, Chee Siang Leow1, Hiromitsu Nishizaki1, Junichi Hoshino2, Takehito Utsuro2 (1.University of Yamanashi, 2.University of Tsukuba)

Keywords:Large Language Models, Customer Service Training

This paper proposes a method for correcting store staff's customer service speech using multiple large language models (LLMs) to improve the quality of the speech of store staff in customer service. In addition, the staff's speech is scored and a commentary is generated to provide a basis for scoring. By correcting the staff's utterances and providing a quantitative evaluation, appropriate feedback can be provided to the staff. This study adopts a roundtable method called ReConcile as one of the methods to utilize multiple LLMs. The results of evaluation experiments showed that by refining the output of multiple LLMs with ReConcile, it was possible to modify the utterances more appropriately than a single LLM and to assign evaluation points that were closer to human senses.

Authentication for paper PDF access

A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.

Password