5:50 PM - 6:10 PM
[2B6-GS-2-02] Representation of Denoising Autoencoders by Tweedie's formula and its Analysis
Keywords:denoising autoencoder, exponential family, Tweedie's formula, feature extraction
Denoising autoencoders learn the score of the data-generating distribution, i.e., ∇log p(x). However, theoretical studies have discussed only the case when the corruption process is Gaussian. To generalize the previous results, in this study, we extend the class of distributions to an exponential family. By using Tweedie’s formula, we show that the generalized DAE learns score of marginal distribution q(˜x) = ∫ q(˜x|x)p(x)dx, while Gaussian DAE learns the score of the data-generating distribution p(x). Then, we focus on the encoder of denoisng autoenoders and investigate what this encoder learn. Numerical experiment shows that this encoder extract features which relate the shape of the data-generating distribution.
Please log in with your participant account.
» Participant Log In