[2K6-ES-2] Machine learning: Modeling

[2K6-ES-2-02] Sparsity enforcement on latent variables for better disentanglement in VAE

A Study on the Latent Space of VAE by Inducing Sparsity in the Encoder Network

〇Paulino Cristovao^1,2, Hidemoto Nakada^2,1, Yusuke Tanimura^2,1, Hideki Asoh² (1. University of Tsukuba, 2. National Institute of Advanced Industrial Science and Technology)

Keywords:disentangle latent representations, latent space, sparse representation, variational autoencoder.

We address the problem of unsupervised latent factorization and reconstruction accuracy. The related work on

unsupervised representations focuses on constraining the second term of Variational Autoencoders loss function:

The Kullback-Leibler component (Beta-VAE, FactorVAE Beta-TCVAE). Despite promising results, this comes with

a trade-off between disentanglement and reconstruction. Besides, it is not clear why minimizing the KL divergence

leads to disentanglement.

In this paper, we propose to achieve disentangled representations by sampling from a sparse distribution. To

give a visual appealing reconstruction for humans, we replace the conventional pixel-wise quadratic by perceptual

loss. We demonstrate the reconstruction quality and disentangled on synthetic datasets.