[1Z2] [General Session] 2. Machine Learning

[1Z2-03] The Study on Stochastic Variational Inference for Topic Modeling with Word Embeddings

〇Kana Ozaki¹, Ichiro Kobayashi¹ (1. Ochanomizu University)

Keywords:Topic Model, Variational Inference, Word Embeddings

Probabilistic topic models based on latent Dirichlet Allocation is widely used to extract latent topics from document collections. In recent years, a number of extended topic models have been proposed, especially Gaussain LDA(G-LDA) has attracted a lot of attention. G-LDA integrates topic modeling with word embeddings by replacing discrete topic distribution over word types with multivariate Gaussian distribution on the word embedding space. This can reflect semantic information into topics. In this paper, we use a G-LDA for our base topic model and apply Stochastic Variational Inference (SVI), an efficient inference algorithm, to estimate topics. This encourages the model to analyze massive document collections, including those arriving in a stream.