[2N4-IS-2c] Machine learning (3/5)

[2N4-IS-2c-02] Styled Comic Portrait Synthesis Based on GAN

Lieu-Hen Chen¹, 〇Yen-Chia Chen¹, Yuh-Ming Huang¹ (1. National Chi Nan University)

Keywords:Comic Portraits, Deep Learning, GANs, NPR, Image Synthesis

Although there are many researches developed for NPR image synthesis using GANs, it is still difficult to create high-quality comic portraits of a real person. Moreover, there are few studies focused on the painting styles of comic artists, though it is the style that make a comic visually unique. And for comic readers, the synthesized comic portraits can be more attractive and meaningful if the portraits are presented in the user-preferred comic styles. Therefore, in this paper, we propose a styled comic portraits synthesis system based on CycleGANs. By integrating Deep Learning and NPR techniques, we aim to transform user’s real pictures into comic portraits with features preserved and defined painting style presented. We first trained a CNN to classify the painting styles of manga artists. Then we trained our GANs with classified and augmented data set, which is generated by mapping comic characters’ 2D texture onto perturbed and deformed 3D facial models. The experiment results shown that the proposed method can successfully create clear and vivid comic portraits, which has a great potential to serve as a useful tool for social network and comic industry.