[1N4-OS-10a] System1型+2型統合AIへの展望(1/2)

[1N4-OS-10a-03] Multi-layer Attentive Text-to-image Generation with CLIP-GAN

〇Yuya Kobayashi¹, Masahiro Suzuki¹, Yutaka Matsuo¹ (1. The University of Tokyo Graduate School of Engineering)

Keywords:Deep Generative Models, Text to Image, Symbol Grounding, artificial creativity

In this research, we are interested in text-to-image from the point of view of pondering. Pondering is a mental activity which update inference results using observations and past experiences. Recently, text-to-image models using CLIP have been gaining much attention for its great generation quality. These models update their latent variables in order to adapt its output to input prompt, and this process can be regarded as a kind of pondering. However, these models update their entire output (latent variable) and they cannot attend to specific areas. Though we said this is a kind of pondering, we also think this process is not structured enough, and it is still more like intuition. In this research, we propose structured models to enable attention to specific areas and update them independently. We hope this improves the generation quality and gives us a clue to thinking about pondering.