Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Statistical inference with deep latent variable models

Abiri, Najmeh LU (2019)
Abstract
Finding a suitable way to represent information in a dataset is one of the fundamental problems in Artificial Intelligence. With limited labeled information, unsupervised learning algorithms help to discover useful representations. One of the applications of such models is imputation, where missing values are estimated by learning the underlying correlations in a dataset. This thesis explores two of unsupervised techniques: stacked denoising autoencoders and variational autoencoders (VAEs). Using stacked denoising autoencoders, we developed a consistent framework to handle incomplete data with multi-type variables. This deterministic method improved missing data estimation compared to several state-of-the-art imputation methods.... (More)
Finding a suitable way to represent information in a dataset is one of the fundamental problems in Artificial Intelligence. With limited labeled information, unsupervised learning algorithms help to discover useful representations. One of the applications of such models is imputation, where missing values are estimated by learning the underlying correlations in a dataset. This thesis explores two of unsupervised techniques: stacked denoising autoencoders and variational autoencoders (VAEs). Using stacked denoising autoencoders, we developed a consistent framework to handle incomplete data with multi-type variables. This deterministic method improved missing data estimation compared to several state-of-the-art imputation methods.

Further, we explored variational autoencoders, a probabilistic form of autoencoders that jointly optimize the neural network-based inference and generative models. Despite the promise of these techniques, the main difficulty is an uninformative latent space. We propose a flexible family, Student's t-distributions, as priors for VAEs to learn a more informative latent representation. By comparing different forms of the covariance matrix for both Gaussian and Student's t-distributions, we conclude that using a weakly informative prior such as the Student's t with a low number of parameters improves the ability of VAEs to approximate the true posterior.

Finally, we used VAEs both with the Gaussian and Student's t-priors as multiple imputation methods on two datasets with missing values. Moreover, with the provided labels on these datasets, we used a supervised network and evaluated the estimation of missing variables. In both cases, VAEs show improvements compared to other methods. (Less)
Abstract (Swedish)
Finding a suitable way to represent information in a dataset is one of the fundamental problems in Artificial Inelegance. With limited labeled information, unsupervised learning algorithms help to discover useful representations. One of the applications of such models is imputation, where missing values are estimated by learning the underlying correlations in a dataset. This thesis explores two of unsupervised techniques: stacked denoising autoencoders and variational autoencoders (VAEs). Using stacked denoising autoencoders, we developed a consistent framework to handle incomplete data with multi-type variables. This deterministic method improved missing data estimation compared to several state-of-the-art imputation methods.... (More)
Finding a suitable way to represent information in a dataset is one of the fundamental problems in Artificial Inelegance. With limited labeled information, unsupervised learning algorithms help to discover useful representations. One of the applications of such models is imputation, where missing values are estimated by learning the underlying correlations in a dataset. This thesis explores two of unsupervised techniques: stacked denoising autoencoders and variational autoencoders (VAEs). Using stacked denoising autoencoders, we developed a consistent framework to handle incomplete data with multi-type variables. This deterministic method improved missing data estimation compared to several state-of-the-art imputation methods.

Further, we explored variational autoencoders, a probabilistic form of autoencoders that jointly optimize the neural network-based inference and generative models. Despite the promise of these techniques, the main difficulty is an uninformative latent space. We propose a flexible family, Student's t-distributions, as priors for VAEs to learn a more informative latent representation. By comparing different forms of the covariance matrix for both Gaussian and Student's t-distributions, we conclude that using a weakly informative prior such as the Student's t with a low number of parameters improves the ability of VAEs to approximate the true posterior.

Finally, we used VAEs both with the Gaussian and Student's t-priors as multiple imputation methods on two datasets with missing values. Moreover, with the provided labels on these datasets, we used a supervised network and evaluated the estimation of missing variables. In both cases, VAEs show improvements compared to other methods. (Less)
Please use this url to cite or link to this publication:
author
supervisor
opponent
  • Professor Winther, Ole, Technical University of Denmark (DTU), Copenhagen, DK
organization
publishing date
type
Thesis
publication status
published
subject
keywords
Deep Learning, Generative Models, Variational Inference, Missing data, Imputation, Fysicumarkivet A:2019:Abiri
publisher
Lund University, Faculty of Science
defense location
Rydbergsalen, Fysicum, Sölvegatan 14A, Lund
defense date
2019-10-31 10:15:00
ISBN
978-91-7895-272-4
978-91-7895-271-7
language
English
LU publication?
yes
id
8f6e3d34-6ca7-4f27-97d7-479ed116a5d6
date added to LUP
2019-10-02 13:06:56
date last changed
2020-12-03 09:09:05
@phdthesis{8f6e3d34-6ca7-4f27-97d7-479ed116a5d6,
  abstract     = {{Finding a suitable way to represent information in a dataset is one of the fundamental problems in Artificial Intelligence. With limited labeled information, unsupervised learning algorithms help to discover useful representations. One of the applications of such models is imputation, where missing values are estimated by learning the underlying correlations in a dataset. This thesis explores two of unsupervised techniques: stacked denoising autoencoders and variational autoencoders (VAEs). Using stacked denoising autoencoders, we developed a consistent framework to handle incomplete data with multi-type variables. This deterministic method improved missing data estimation compared to several state-of-the-art imputation methods. <br/><br/>Further, we explored variational autoencoders, a probabilistic form of autoencoders that jointly optimize the neural network-based inference and generative models. Despite the promise of these techniques, the main difficulty is an uninformative latent space. We propose a flexible family, Student's t-distributions, as priors for VAEs to learn a more informative latent representation.  By comparing different forms of the covariance matrix for both Gaussian and Student's t-distributions, we conclude that using a weakly informative prior such as the Student's t with a low number of parameters improves the ability of VAEs to approximate the true posterior.<br/><br/>Finally, we used VAEs both with the Gaussian and Student's t-priors as multiple imputation methods on two datasets with missing values. Moreover, with the provided labels on these datasets, we used a supervised network and evaluated the estimation of missing variables. In both cases, VAEs show improvements compared to other methods.}},
  author       = {{Abiri, Najmeh}},
  isbn         = {{978-91-7895-272-4}},
  keywords     = {{Deep Learning; Generative Models; Variational Inference; Missing data; Imputation; Fysicumarkivet A:2019:Abiri}},
  language     = {{eng}},
  publisher    = {{Lund University, Faculty of Science}},
  school       = {{Lund University}},
  title        = {{Statistical inference with deep latent variable models}},
  year         = {{2019}},
}