[4Xin1] Poster session 2

[4Xin1-11] Assessing Few-shot Counter-Arguments Generation via Large Language Models

〇Taisei Ozaki¹, Chihiro Nakagawa^1,5, Shohichi Naito^2,3, Naoya Inoue^4,5, Kenshi Yamaguchi³, Atsuhiko Shintani¹ (1.Osaka Prefecture University, 2.RICOH COMPANY,LTD., 3.Tohoku University, 4.Japan Advanced Institute of Science and Technology, 5.Institute of Physical and Chemival Research)

Keywords:NLP, AI, Argument Evaluation, Text generation, LLMs

This study explores the potential of using GPT-3, a large language model, to generate effective counter arguments that can aid educators in cultivating critical thinking skills. We examine the quality of the generated texts and the possibility of improving their quality. Using online debate forums, we collected counter arguments, and compared them with generated ones. We found that the quality of the latter were the same or even higher in terms of the correctness of the logic of the counter argument and its relevance to the initial argument. The high degree of similarity between the generated and collected counter arguments on BERTScore suggests that GPT-3 can assist educators in providing appropriate counterargument examples. The study reports the promising use of GPT-3 in improving the quality of educational materials and enhancing the learning experience.