[3Xin2] Poster session 1

[3Xin2-62] Comparison of multiple methods in predicting related laws

〇Sakiho Nakashita¹, Eitetsu Togo¹, Hideaki Kikuchi¹, Shohei Fujikura², Riu Noritake² (1.Univ. of Waseda, 2.LawFlow, Inc.)

Keywords:Legal AI, LLM, QA

With the development of Large Language Models (LLMs), it is expected that legal technologies, such as what is called Legal Tech. In this study, we compared and verified the prediction accuracy of relevant laws for legal consultation using multiple LLMs. GPT-3.5 and GPT-4 were set as the baseline, and a model fine-tuning GPT-3.5 with the legal consultation data set was constructed as the proposed model. The accuracy of each model was compared, and the F1 Score was 0.44 for GPT-3.5, 0.57 for GPT-4, and 0.69 for the proposed model. Comparison and analysis of the prediction results of each model confirmed that the proposed model was able to correctly answer questions that the baseline had mispredicted, indicating the effectiveness of the proposed model.