Variational Bayes in Private Settings (VIPS) (Extended Abstract)
Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence
Journal track. Pages 5050-5054. https://doi.org/10.24963/ijcai.2020/705
Many applications of Bayesian data analysis involve sensitive information such as personal documents or medical records, motivating methods which ensure that privacy is protected. We introduce a general privacy-preserving framework for Variational Bayes (VB), a widely used optimization-based Bayesian inference method. Our framework respects differential privacy, the gold-standard privacy criterion. The iterative nature of variational Bayes presents a challenge since iterations increase the amount of noise needed to ensure privacy. We overcome this by combining: (1) an improved composition method, called the moments accountant, and (2) the privacy amplification effect of subsampling mini-batches from large-scale data in stochastic learning. We empirically demonstrate the effectiveness of our method on LDA topic models, evaluated on Wikipedia. In the full paper we extend our method to a broad class of models, including Bayesian logistic regression and sigmoid belief networks.
Multidisciplinary Topics and Applications: Security and Privacy
Uncertainty in AI: Approximate Probabilistic Inference
Machine Learning: Probabilistic Machine Learning
Natural Language Processing: Natural Language Processing