[2Win5] Poster session 2

[2Win5-23] Analysis of Internal Representations of Knowledge with Expressions of Familiarity

〇Kenshiro Tanaka¹, Yoshihiro Sakai¹, Yufeng Zhao¹, Naoya Inoue^1,2, Kai Sato³, Ryosuke Takahashi³, Benjamin Heinzerling^2,3, Kentaro Inui^4,3,2 (1.Japan Advanced Institute of Science and Technology, 2.Institute of Physical and Chemical Research, 3.Tohoku University, 4.Mohamed bin Zayed University of Artificial Intelligence)

Keywords:language models, knowledge representation, familiarity judgement

Research on the ability of large language models (LLMs) to judge the familiarity of knowledge is progressing. However, little attention has been given to whether LLMs can assess the familiarity of knowledge when linguistic expressions such as "It is known that..." are included during training. This study investigates how familiarity is internally represented in LLMs. To achieve this, we trained the models on descriptions of knowledge accompanied by linguistic expressions indicating familiarity. The internal representations of the models were then analyzed. The findings reveal that (1) familiarity information is separately retained in the internal representations of knowledge, for the linguistic expressions provided during training, and (2) familiarity information is separately maintained for each position of the linguistic expressions. This study provides a foundation for understanding the mechanisms underlying LLMs' ability to judge familiarity.