[2D1-GS-9] Natural language processing, information retrieval: Support technology

[2D1-GS-9-03] Generation Control Methods for Reliable Feedback Comment Generation

〇Kazuaki Hanawa^1,2, Ryo Nagata^3,4, Kentaro Inui^2,1 (1. RIKEN, 2. Tohoku University, 3. Konan University, 4. JST, PRESTO)

Keywords:Natural Language Processing, Grammatical Error, Confidence Estimation

Feedback comment generation is the task of generating comments on writing techniques to help learners improve their writing skills. Although neural-based generation methods are promising, their generation abilities are so powerful that they often generate plausible, but inappropriate feedback comments as in {\em The verb ``go'' is a transitive verb, and thus does not take a preposition before its object.\/} These plausible, inappropriate generation results are likely to harm learning. With this in mind, this paper explores methods for estimating generation reliability to filter out false generation results. To be precise, it compares four types of reliability measures based on cosine similarity, generation probability, and actual and predicted edit rates. Experiments show that reliability measures based on cosine similarity and predicted edit rate are superior to the other two. It further looks into the experimental results, showing in which case the two superior measures perform better.