Student-t Variational Autoencoder for Robust Density Estimation

Student-t Variational Autoencoder for Robust Density Estimation

Hiroshi Takahashi, Tomoharu Iwata, Yuki Yamanaka, Masanori Yamada, Satoshi Yagi

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 2696-2702. https://doi.org/10.24963/ijcai.2018/374

We propose a robust multivariate density estimator based on the variational autoencoder (VAE). The VAE is a powerful deep generative model, and used for multivariate density estimation. With the original VAE, the distribution of observed continuous variables is assumed to be a Gaussian, where its mean and variance are modeled by deep neural networks taking latent variables as their inputs. This distribution is called the decoder. However, the training of VAE often becomes unstable. One reason is that the decoder of VAE is sensitive to the error between the data point and its estimated mean when its estimated variance is almost zero. We solve this instability problem by making the decoder robust to the error using a Bayesian approach to the variance estimation: we set a prior for the variance of the Gaussian decoder, and marginalize it out analytically, which leads to proposing the Student-t VAE. Numerical experiments with various datasets show that training of the Student-t VAE is robust, and the Student-t VAE achieves high density estimation performance.
Keywords:
Machine Learning: Machine Learning
Machine Learning: Neural Networks
Machine Learning: Unsupervised Learning
Machine Learning: Deep Learning
Machine Learning: Probabilistic Machine Learning