[3D5-GS-2] Machine learning

[3D5-GS-2-02] Comment generation using a large-scale language model for fashion item recommendation

〇Yugen Sato¹, Sayaka Aoba¹, Ryozo Masukawa¹, Makoto Sato¹, Shosuke Haji¹, Taiga Matsui², Keita Ishikawa², Tomohiro Takagi¹ (1. Meiji University, 2. airCloset, Inc.)

Keywords:Large-scale model, Fashion Item Recommendation, comment generation, Vision&Language

In personal styling, the stylist selects fashion items based on the client's characteristics, purpose of use, season, and various other factors. The stylist then carefully comments on the reasons for the selection and sends it to the customer along with the recommended fashion item. In response to this, we use MAGMA, a method that supports multimodal input of language models by means of adapter-based fine tuning, to construct a model that generates comment suggestions based on the combination of item images and prompts. We conducted quantitative and qualitative evaluations of the proposed model and confirmed that the model using MAGMA is superior to the conventional method.