[4D3-GS-6] Language media processing: applications

[4D3-GS-6-01] A method of selecting a sentence with puns reflecting what is in the picture

〇Reki Asano¹, Motoki Yatsu¹, Takeshi Morita¹ (1. Aoyama Gakuin University)

Keywords:Image captioning, Humor

When a social robot makes an utterance based on the surrounding situation obtained from image input, it is considered that humor such as puns can improve its entertainment aspect. Therefore, we propose a ranking method for selecting puns. In the proposed method, plain captions are generated from the Japanese caption generation model learned from the STAIR Captions dataset, and important words and other morphemes are extracted from the obtained captions. The words obtained in this way are weighted so that the words resulting from object detection and important word extraction have larger values. As the output, the method selects a pun sentence in pun database that maximizes sum of the weights. In the subjective evaluation experiment, the proposed method selected puns for each of 10 images randomly selected from the MS COCO dataset. We asked 10 subjects whether the puns seem appropriate for the images and to make 5-point evaluations. As a result, the average evaluation value was 3.11 for the question, which was slightly higher than "neither".