JSAI2022

Presentation information

Interactive Session

General Session » Interactive Session

[3Yin2] Interactive session 1

Thu. Jun 16, 2022 11:30 AM - 1:10 PM Room Y (Event Hall)

[3Yin2-23] Image Captioning that Reflects the Intent of the Explainer based on Tracing with a Pen.

〇Sayako Watanabe1, Ichiro Kobayashi1 (1.Ochanomizu University)

Keywords:captioning, sentence length control, control signal

In recent years, research on image caption generation has evolved to include not only the generation of image captions based on information obtained from image preprocessing, but also the generation of captions based on the user's interest in the image by providing additional information corresponding to the viewpoint, called control signals, to the image processing information. In this paper, we propose a new method to generate captions based on the user's interests.
In general, when people describe the image, they usually use their fingers to trace the object they want to describe.
In this study, we consider tracing the image as a control signal.
And, we propose an interactive generating image caption method that is more in line with the explainer by reflecting the meaning of the traces.

Authentication for paper PDF access

A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.

Password