[1J5-OS-10c] OS-10

[1J5-OS-10c-05] Applying Generative AI to Drafting of Governmental Answers for Debate in the National Diet of Japan

A Case Study Focusing on Diet Debate Regarding Diplomacy vis-a-vis North Korea from 2003 to 2005

〇Takeo Harada^1,2,3 (1. Institute for International Strategy and Information Analysis, Inc. , 2. The Social Cooperation Program: Mathematical Engineering of Morality Emotions, Graduate School of Engineering, University of Tokyo, 3. Gakushuin Women's College)

Keywords:LLM, Generative AI, Politics, Diplomacy, North Korea

In the midst of declining population, the Japanese bureaucracy is currently urged to maximize its efficiency. In this regard, the Government of Japan once stated it would begin to study the utilization of generative AI for drafting governmental answers for Diet debate, of which any outcome hasn’t been disclosed yet. While making use of LangChain, gpt-3.5-turbo and Web API of full-text database for the minutes of the Japanese Diet, the author, who was personally in charge of North Korean affairs from January 2003 to March 2005 in the Japanese diplomatic service, compares the generated texts by the drafting system based on LLM and the minutes of the governmental answers to the same questions on diplomatic issues of North Korean affairs delivered in the Diet debates in the same period. The BERTScore turns out to remain below 0.7, while the incapability of the drafting system using the generative AI to express minute nuances needed in governmental answers to questions regarding such issues as North Korea is quite obvious based on human evaluation. This is both partly and technically rooted in typical structures of generative AI, which still limits further improvement of efficiency in the Japanese bureaucracy, particularly diplomatic service.