[2L5-GS-1] Fundamental AI, theory, algorithm:

[2L5-GS-1-04] Development of Structural Learning Methods for Language Models Using Algebraic Statistical Approaches

〇Akihiro Maeda^1,3, Takuma Torii², Shohei Hidaka¹ (1. Japan Advanced Institute of Science and Technology, 2. Tokyo Denki University, 3. JSPS)

Keywords:Language model, Algebraic statistics, Model selection

Algebraic statistics, a fusion of algebraic geometry and statistics, represents probabilistic models as low-dimensional structures in high-dimensional spaces and captures their structures through ideals of polynomial rings. From this perspective, it is expected to naturally model complex constraints such as conditional independence and hierarchical structures in linguistic data. This study formulates probabilistic models of language as joint probability distributions of words composing sentences and applies algebraic statistical approaches. By characterizing the models through vanishing ideals that constrain the probabilistic structures, the study develops methods to select models that fit the data. Experiments using synthetic data demonstrated that the proposed method achieved performance comparable to information criteria and directly extracted the structural characteristics of the probabilistic model.