[3Rin4] Interactive 1

[3Rin4-90] Verification of the Effectiveness of Introducing Latent Topics into Text Summarization Model with Pretrained Encoders

〇Kana Ozaki¹, Ichiro Kobayashi¹ (1.Ochanomizu University)

Keywords:text summarization, pre-trained language model, topic model

In recent years, the necessity of text summarization that makes a summary of documents have increased because of having to deal with a numerous number of documents. Motivated by the remarkable development in deep learning, the studies of text summarization are developing dramatically. Among them, BERTSUM employing pretrained language model BERT for summarization achieved state-of-the-art results in both extractive and abstractive summarization. On the other hand, some summarization models outperform other models by adding the latent topic information of the target documents to summarize. In this paper, we propose a model expanding BERTSUM in terms of being able to use the latent topics extracted from source documents for BERT tokens, and verify the effectiveness of our model through the experiments for extractive summarization. Moreover, we analyze the results to verify whether the topic information works to make a better summary by measuring the coverage of topic distributions between system output and gold summary, and the coherence of topic in system output.