[4A3-GS-10] AI application:

[4A3-GS-10-02] Development of Prompt Attack Data Collection Application for LLMs and Analysis of Collected Data Characteristics

〇Yuta Hayashi¹, Yusuke Ishiguro², Tasuku Sasaki³, Sekine Satoshi⁴ (1. Determinant, Inc., 2. makery, inc., 3. The University of Tokyo, Graduate School of Information Science and Technology, 4. NII LLMC)

[[Online]]

Keywords:AI Safety, Jailbreak, Prompt Injection

As Large Language Models become increasingly widespread, countermeasures against attack methods such as jailbreaking and prompt injection have become an urgent issue. Existing defense methods like Safeguard Models, including Llama Guard, have been found to perform inadequately against attacks in Japanese. In this research, we developed AILBREAK, an attack dataset collection application utilizing gamification to enhance LLM defense capabilities against Japanese-language prompt attacks. This application implements a mechanism that collects manually crafted attack prompts from users, featuring stages composed of various challenges based on safety categories, such as extracting passwords from enemies through battle-game elements. The design achieves both educational impact and data collection objectives. The collected dataset will be made publicly available for improving LLM defense functions and developing Japanese language-specific Safeguard Models. This paper reports on the application design, data collection methodology, and characteristics of the collected data.