Advanced

Establishing strong imputation performance of a denoising autoencoder in a wide range of missing data problems

Abiri, Najmeh LU ; Linse, Björn LU ; Edén, Patrik LU and Ohlsson, Mattias LU (2019) In Neurocomputing
Abstract

Dealing with missing data in data analysis is inevitable. Although powerful imputation methods that address this problem exist, there is still much room for improvement. In this study, we examined single imputation based on deep autoencoders, motivated by the apparent success of deep learning to efficiently extract useful dataset features. We have developed a consistent framework for both training and imputation. Moreover, we benchmarked the results against state-of-the-art imputation methods on different data sizes and characteristics. The work was not limited to the one-type variable dataset; we also imputed missing data with multi-type variables, e.g., a combination of binary, categorical, and continuous attributes. To evaluate the... (More)

Dealing with missing data in data analysis is inevitable. Although powerful imputation methods that address this problem exist, there is still much room for improvement. In this study, we examined single imputation based on deep autoencoders, motivated by the apparent success of deep learning to efficiently extract useful dataset features. We have developed a consistent framework for both training and imputation. Moreover, we benchmarked the results against state-of-the-art imputation methods on different data sizes and characteristics. The work was not limited to the one-type variable dataset; we also imputed missing data with multi-type variables, e.g., a combination of binary, categorical, and continuous attributes. To evaluate the imputation methods, we randomly corrupted the complete data, with varying degrees of corruption, and then compared the imputed and original values. In all experiments, the developed autoencoder obtained the smallest error for all ranges of initial data corruption.

(Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
epub
subject
keywords
Autoencoder, Deep learning, Imputation, Missing data
in
Neurocomputing
publisher
Elsevier
external identifiers
  • scopus:85069939556
ISSN
0925-2312
DOI
10.1016/j.neucom.2019.07.065
language
English
LU publication?
yes
id
f10ae285-9eb9-41b5-a2d2-bca90974ddd8
date added to LUP
2019-08-26 15:31:06
date last changed
2019-11-25 09:33:44
@article{f10ae285-9eb9-41b5-a2d2-bca90974ddd8,
  abstract     = {<p>Dealing with missing data in data analysis is inevitable. Although powerful imputation methods that address this problem exist, there is still much room for improvement. In this study, we examined single imputation based on deep autoencoders, motivated by the apparent success of deep learning to efficiently extract useful dataset features. We have developed a consistent framework for both training and imputation. Moreover, we benchmarked the results against state-of-the-art imputation methods on different data sizes and characteristics. The work was not limited to the one-type variable dataset; we also imputed missing data with multi-type variables, e.g., a combination of binary, categorical, and continuous attributes. To evaluate the imputation methods, we randomly corrupted the complete data, with varying degrees of corruption, and then compared the imputed and original values. In all experiments, the developed autoencoder obtained the smallest error for all ranges of initial data corruption.</p>},
  author       = {Abiri, Najmeh and Linse, Björn and Edén, Patrik and Ohlsson, Mattias},
  issn         = {0925-2312},
  language     = {eng},
  month        = {07},
  publisher    = {Elsevier},
  series       = {Neurocomputing},
  title        = {Establishing strong imputation performance of a denoising autoencoder in a wide range of missing data problems},
  url          = {http://dx.doi.org/10.1016/j.neucom.2019.07.065},
  doi          = {10.1016/j.neucom.2019.07.065},
  year         = {2019},
}