[1S4-GS-2] Machine learning:

[1S4-GS-2-02] Evaluating the Effectiveness of Model Linearization in Task Analogies

Kotaro Yoshida¹, 〇Takafumi Horie², Yuji Naraki, Ryotaro Shimizu^3,4, Ryosuke Yamaki^2,5, Hiroki Naganuma^6,7,5 (1. Institute of Science Tokyo, 2. Ritsumeikan University, 3. ZOZO research, 4. University of California San Diego, 5. Proplace Inc., 6. Mila, 7. University of Montreal)

Keywords:Task Arithmetic, Task Analogies, Finetuning

This study examines the effectiveness of model linearization in enhancing task analogies in model editing. task analogies aim to reproduce analogy structures in parameter space, similar to those observed in Word2Vec, and leverage models fine-tuned on different tasks to enable efficient model editing for adaptation to new tasks. However, it is known that task analogies based on conventional fine-tuning do not achieve high performance. A key challenge identified in prior research is the nonlinear relationship between function space and parameter space, which prevents the preservation of analogy structures in function space when they exist in parameter space. In this study, we apply model linearization to task analogies and investigate the transferability of these analogy structures to function space.