[3B3-E-2] Machine learning: image recognition and generation

[3B3-E-2-03] Cycle Sketch GAN: Unpaired Sketch to Sketch Translation Based on Cycle GAN Algorithm

〇Takeshi Kojima¹ (1. Peach Aviation Limited)

Keywords:Cycle GAN, sketch drawing generative model, unsupervised learning, Transformer, QuickDraw

Unlike pixel image generation, sketch drawing generative model outputs a sequence of pen stroke information. This paper proposes Cycle Sketch GAN: the first model that learns to translate a sketch drawing from source domain to target domain in the absence of paired dataset. Based on Cycle GAN algorithm, this model uses Transformer Encoder architecture in generators. Transformer Encoder feeds the input stroke information in source domain and generates the parameters for output distribution, from which the stroke information in target domain is sampled by reparameterization trick. The negative log likelihood of the distribution is used as cycle consistency loss. This model is trained and evaluated by some QuickDraw datasets. Qualitative evaluation shows that this model can practically translate sketch drawings from source domain to target domain. Quantitative evaluation by user study showed that 42 % of the translated sketches is recognizable compared to 71 % of the human sketches.