[1M3-OS-47a] OS-47

[1M3-OS-47a-05] De-Tuning of Large Language Models

〇Koki Iwai¹, Yusuke Kumagae¹, Yukino Baba² (1. Hakuhodo DY Holdings Inc., 2. The University of Tokyo)

Keywords:Large Language Model, Benchmark, Persona, Role-Playing

Large Language Models (LLMs) possess the ability to perform well on unknown tasks and flexibly alter their behavior according to prompts. Leveraging this characteristic, there are attempts to assign virtual personas or personalities to LLMs and make them behave accordingly. If we could intentionally limit LLM performance, the constructed virtual personas would likely become more realistic (e.g., making a kindergartener unable to solve integral calculus). This paper addresses such intentional performance degradation of LLMs. Using multiple Japanese benchmark tasks, we report that it is difficult to degrade LLM performance in downstream tasks through prompts alone. We also examine the benchmarks necessary for measuring performance degradation.