[1D2-OS-3a] ニュースメディアのデータサイエンス(1/3)

[1D2-OS-3a-03] News Articles Summarization with MMR Sentence Selection and TF-IDF Sentence Compression

〇Shotaro Ishihara¹, Norihiko Sawa¹ (1. Nikkei Inc.)

Keywords:natural language processing, extractive summarization, MMR, TF-IDF

This paper proposes a method to summarize news articles by sentence selection and compression. We can extract N texts which represent the article, and enumerate summary candidates by compressing each text through syntactic analysis. MMR (Maximal Marginal Relevance) and TF-IDF (Term Frequency - Inverse Document Frequency) are used as metrics. Experiments showed that the proposed method was able to extract the same topics as the human editor's summary in the rate of 26%. Even though the rate wasn't high enough, most of the achievements couldn't be described as incorrect as one of the summary candidates. This methodology has a potential to reduce the burden on editors and generate some collaboration.