[2H6-GS-9] Natural language processing, information retrieval: Document generation

[2H6-GS-9-04] A pseudo-problem creation method for transfer learning of sentence elimination probelms

Hiroki Inoue¹, 〇Seiki Matoba¹, Hiromi Narimatsu², Hiroaki Sugiyama², Ryuichiro Higashinaka², Hirotoshi Taira¹ (1. Osaka Institute of Technology, 2. NTT Communication Science Laboratories)

Keywords:Sentence Elimination Problem, Machine Reading Comprehension, Can a Robot Get into the University of Tokyo?, Natural Language Processing

In the “Can a Robot get into the University of Tokyo?” project, we tackle the problem of sentence elimination in English. In the previous research, we proposed a simple automatically pseudo-problem generation method for fine-tuing BERT. It remains lower than. To improve the accuracy, we propose methods applying XLNet or RoBERTa instead of BERT for pseudo-problem generation. We examined how the accuracy changes depending on the method of extracting and inserting unnecessary sentences. As a result, when extraction and insertion were performed randomly, the system has achived the highest accuracy rate using XLNet or RoBERTa. In addition, the accuracy was higher 19 points than that of BERT.