[3L6-OS-32] OS-32

[3L6-OS-32-03] Formation of Geospatial Representations in Large Language Models and the Effect of Training Data

〇Hiroto Otake^1,2, Hiroki Ouchi^1,3, Shintaro Ozaki^1,2, Tatsuya Hiraoka⁴, Taro Watanabe¹, Yusuke Miyao^5,2, Yohei Oseki⁵, Yu Takagi² (1. NARA Institute of Science and Technology, 2. National Institute of Informatics Research and Development Center for Large Language Models, 3. Institute of Physical and Chemical Research, 4. Mohamed bin Zayed University of Artificial Intelligence, 5. The University of Tokyo)

Keywords:Explainability of Artificial Intelligence, Artificial Intelligence Knowledge Assessment, Geographic Domain

Large language models (LLMs) have demonstrated the ability to solve tasks in geographic domains, and it has been suggested that these capabilities rely on an internal geospatial world model. However, previous studies have mainly examined such representations using only a small number of the models trained on English-centric data, leaving it unclear how geospatial representations emerge in some models trained on other languages. In this study, we investigate the internal geographic representations of multiple regions in models pre-trained on data in different languages. Our experimental results indicate that the properties of these world models may strongly depend on the language used during training.