Presentation information

Interactive Session

[3Rin4] Interactive 1

Thu. Jun 11, 2020 1:40 PM - 3:20 PM Room R01 (jsai2020online-2-33)

[3Rin4-83] A Conditional Language Model for Controlling Sentence Length

〇Koichi Nagatsuka1, Masayasu Atsumi1 (1.Soka University)

Keywords:Conditional Language Model, Controlling Sentence Length, Sentence Generation

The performance of sentence generation task has been dramatically improved due to the development of a pre-trained language model in recent years. In particular, a pre-trained conditional language model such as GPT-2 shows the nearly human-level performance of generating sentences by self-supervised learning. Furthermore, a conditional language model such as CTRL can successfully control the topics and styles of the generated text with control codes. However, it is effective to input a control code in a form of continuous representations rather than discrete representations in case of the sentence generation task. In this study, we propose an approach for the controllable sentence generation with a desirable length by explicitly adding a distributed representation of a target length. We use the positional encoding to obtain the continuous representation of a target length, and fine-tuned a pre-trained GPT-2 with wikitext-103. The result shows our approach is effective for controlling sentence length while generating natural sentences.

Authentication for paper PDF access

A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.