[1O4-GS-7] Vision, speech media processing

[1O4-GS-7-05] The image recommendation model for question text on Q&A which is based on Sentence-BERT

〇Yasuhito Fujisawa¹, Haruka Yamashita¹ (1. Sophia University)

Keywords:Image Recommendation, Sentence BERT

In question-and-answer sessions on Q&A services, it is sometimes difficult to read a long or poorly-written question and answer. In such cases, if the system can recommend appropriate images for the questions, it can assist reading comprehension based on the information in the images. In this study, we propose a machine learning model for recommending appropriate images for questions in a Q&A service using Sentence-BERT (SBERT). Specifically, the model achieves this by converting question sentences and image captions into a vector using SBERT, measuring the cosine similarity between them, and recommending the image with the caption that has the maximum value. From a practical point of view, it is also necessary to minimize inappropriate recommendation results when SBERT malfunctions. Therefore, in order to ensure that the recommended images are at least correctly recommended from the categorical point of view, a categorization model based on BERT's transfer learning is applied as an auxiliary. This is achieved by classifying the recommended images into categories that exist in each Q&A service and performing SBERT and cosine similarity measures within each category.