[2B4-GS-6] Language media processing: language model

[2B4-GS-6-01] Automatic Literature Review Generation with Transformer Encoder-Decoder Model

〇Tetsu Kasanishi¹, Masaru Isonuma¹, Junichiro Mori^1,2, Ichiro Sakata¹ (1. The University of Tokyo, 2. RIKEN)

Keywords:Natural Language Processing, Document summarization, Transformer

In this study, we aim to automatically generate survey papers by the Transformer encoder-decoder model to assist researchers in their surveys. As a preliminary effort, we attempted to generate chapters of survey papers by the existing Transformer encoder-decoder model, Longformer and Big Bird. As a result, we find some results where Big Bird correctly captures the topic to be explained and generate appropriate sentences. On the other hand, the model generates inappropriate sentences for the most part since it could not correctly determine the topic.