[2B4-GS-6] Language media processing: language model

[2B4-GS-6-02] Dynamic Structured Neural Topic Model with Self-attention Mechanism

〇Nozomu Miyamoto¹, Masaru Isonuma¹, Junichiro Mori^1,2, Ichiro Sakata¹ (1. The University of Tokyo, 2. Riken AIP)

Keywords:topic model, self-attention

In this study, we aim to create a topic model that takes into account time-series transitions of topics and multi-year dependencies between topics. As a part of this effort, we show the limitations of the existing model, Dynamic Embedded Topic Model (D-ETM) and propose Dynamic Structured Neural Topic Model (DSNTM). DSNTM is based on D-ETM, while introducing a self-attention mechanism to represent the relationship between topics. After explaining the specific architecture of DSNTM, we discuss the current challenges and future prospects of our study.