[2K4-ES-2] Machine learning: GAN

[2K4-ES-2-05] Investigating Conditional CycleGAN for Real-World Photo to Conditional Artistic Image Translation

〇Rina Komatsu¹, Tad Gonsalves¹ (1. Sophia University)

Keywords:Style Transfer, Conditional CycleGAN, Deep Learning, Adversarial Learning

The unpaired image-to-image translation architecture “CycleGAN” demonstrates photo-to-artistic image translation. This translation is one-to-one domain translation: from the content domain of real-world images to the style domain of the specific artist’s style. If we want to obtain the different styles of artists, multiple CycleGANs are needed and each of them need to learn each of the styles.
To be able to translate multi styles using a single Generator, our study proposed one-to-N domain translation architecture based on Conditional CycleGAN expanded from CycleGAN. By inputting the content image and the conditional vector which relates to the 7 kinds of artistic styles, we could get the artistic images such as Monet style and Gogh style.
When calculating Adversarial Loss for style data, we set and compared 2 different methods. The first is the loss through PatchGAN which is obtained by discriminating each patch of Discriminator’s output. The second is the summation of PatchGAN and Auxiliary Classifier Loss.
After training each Generator and Discriminator in different learning architectures, the Conditional CycleGAN architecture with PatchGAN could demonstrate the same conditional artistic translation as the one with PatchGAN and Auxiliary Classifier Loss. Each Generator could output the artistic images which hold unique style features of the artist such as drawings and touches.