[3T1-GS-6] Language media processing

[3T1-GS-6-02] Verification of Chain-of-Thought Prompting in Japanese

〇Kaito Horio¹, Eiki Murata¹, Hao Wang¹, Tatuya Ide¹, Daisuke Kawahara¹, Takato Yamazaki², Kenta Shinzato², Akifumi Nakamachi², Shengzhe Li², Toshinori Sato² (1. Waseda University, 2. LINE Corporation)

[[Online]]

Keywords:NLP

Foundation models can be adapted to various tasks by Few-Shot learning, which uses a small number of examples as a prompt. To improve Few-Shot learning, Chain-of-Thought (CoT) prompting has been proposed, which divides the process of thinking into steps. Although the effectiveness of CoT has been proved in English tasks requiring logical reasoning, it has not been verified in Japanese. We examine the effectiveness of CoT in Japanese using a Japanese foundation model, HyperCLOVA JP. We first construct Japanese datasets for the following three tasks: arithmetic, commonsense, and symbolic reasoning. Then, we conduct experiments using HyperCLOVA models of four different sizes. The results showed that CoT prompts were more accurate than standard prompts, and that the performance of CoT prompts was correlated with model size.