[4A1-GS-10] AI application:

[4A1-GS-10-02] Classification of Treatment Discontinuation Reasons in Japanese Electronic Health Records Using Large Language Models

〇Takeshi Ito¹, Tomohide Yoshie², Sohei Yoshimura², Nobuyuki Ohara³, Shuntaro Yada⁴, Shoko Wakamiya¹, Eiji Aramaki¹ (1. Nara Institute of Science and Technology, 2. National Cerebral and Cardiovascular Center, 3. Kobe City Medical Center General Hospital, 4. University of Tsukuba)

Keywords:Large Language Models, Adverse Event Detection, Japanese EHRs, Real World Data

A large volume of free-text data in electronic health records (EHRs) describes treatment discontinuations, including those caused by adverse events. However, because this information is insufficiently structured in existing databases and thus difficult to extract, it remains underutilized despite its significant value. In this study, we combined automated labeling using Large Language Models (LLMs) with a small amount of manual annotation to efficiently classify treatment discontinuations due to adverse events. We integrated approximately 6,256 LLM-labeled records with 200 manually annotated samples, then fine-tuned JMedRoBERTa and T5. When evaluated on a 100-record test set, the T5 model demonstrated high precision (0.83) but was limited to a recall of 0.25. Missing adverse events is a critical concern in clinical practice, underscoring the need for more extensive training data. In the future, we plan to expand our approach to other discontinuation reasons (e.g., patient preferences or insufficient therapeutic effect) and strive for practical clinical implementation.