[3F4-OS-42a] OS-42

[3F4-OS-42a-02] "Negative In-context Learning for Mitigating Copyright Infringement"

〇Satoru Utsunomiya¹, Masaru Isonuma^1,2,3, Junichiro Mori^1,4, Ichiro Sakata¹ (1. The University of Tokyo, 2. The University of Edinburgh, 3. National Institute of Informatics, 4. RIKEN Center for Advanced Intelligence Project)

Keywords:LLM, Incontext learning, contrastive decoding

This study introduces a novel unlearning technique to address the unauthorized reproduction of copyrighted materials by large language models (LLMs). Although unlearning techniques have recently been introduced as an efficient, low-cost solution for addressing copyright infringement, they require access to model parameters and are therefore not applicable to black-box LLMs.

In this study, we propose negative in-context learning, an unlearning method that can be applied for black-box LLMs based on in-context learning. In-context learning allows LLMs to learn knowledge given a few examples without access to model parameters. In contrast, negative in-context learning makes LLM unlearn knowledge by providing negative in-context examples made by using contrastive decoding. By learning these negative in-context examples, LLMs can selectively forget specific knowledge without updating model parameters.

Experimental results show that the introduction of negative in-context examples leads to a significant decrease in BLEU, Jaccard, and ROUGE-L scores, confirming that our method effectively interferes with the model’s recall of the original information.