[2L1-GS-2] Machine learning: NLP

[2L1-GS-2-02] Adaptation to a question answering task using GPT-2

〇Kazunori Yawata¹, Keisuke Kiryu¹, kota katayanagi¹, Ken Mohri², Kazuho Sekimoto² (1. The Dai-ichi-Life Infomation Systems Co.,Ltd, 2. Deloitte Touche Tohmatsu LLC)

Keywords:Natural Language Processing, Pretrained Language Model, GPT-2

In recent years, there has been a remarkable development in natural language processing technology using deep learning algorithms, such as BERT developed by Google and the GPT-x series developed by the OpenAI Foundation. Nowadays, research is being conducted not only on simple tasks such as categorizing sentences, but also on generative tasks such as creating and summarizing sentences. In this experiment, we generated a pre-training model of GPT-2 and fine-tuned it to adapt to the question-answering task in order to verify whether GPT-2 can be applied to question-answering chatbots. For fine-tuning, we used the FAQ data of a life insurance company. As a result, we were able to obtain natural answers in about 80% of the test data and ideal answers in about 60%. We believe that this mechanism can be used to configure a question and answer system with a different approach from the rule-based system.