Advanced

Comparison of standard resampling methods for performance estimation of artificial neural network ensembles

Green, Michael LU and Ohlsson, Mattias LU (2007) Third International Conference on Computational Intelligence in Medicine and Healthcare In Third International Conference on Computational Intelligence in Medicine and Healthcare
Abstract
Estimation of the generalization performance for classification within the medical applications domain is always an important task. In this study we focus on artificial neural network ensembles as the machine learning technique. We present a numerical comparison between five common resampling techniques: k-fold cross validation (CV), holdout, using three cutoffs, and bootstrap using five different data sets. The results show that CV together with holdout $0.25$ and $0.50$ are the best resampling strategies for estimating the true performance of ANN ensembles. The bootstrap, using the .632+ rule, is too optimistic, while the holdout $0.75$ underestimates the true performance.
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
performance estimation, k-fold cross validation, bootstrap, artificial neural networks
in
Third International Conference on Computational Intelligence in Medicine and Healthcare
editor
Ifeachor, Emmanuel
pages
6 pages
conference name
Third International Conference on Computational Intelligence in Medicine and Healthcare
language
English
LU publication?
yes
id
06a42779-0a76-4c80-8d24-84f384f01135 (old id 593195)
date added to LUP
2007-11-05 15:13:54
date last changed
2016-04-16 12:24:24
@misc{06a42779-0a76-4c80-8d24-84f384f01135,
  abstract     = {Estimation of the generalization performance for classification within the medical applications domain is always an important task. In this study we focus on artificial neural network ensembles as the machine learning technique. We present a numerical comparison between five common resampling techniques: k-fold cross validation (CV), holdout, using three cutoffs, and bootstrap using five different data sets. The results show that CV together with holdout $0.25$ and $0.50$ are the best resampling strategies for estimating the true performance of ANN ensembles. The bootstrap, using the .632+ rule, is too optimistic, while the holdout $0.75$ underestimates the true performance.},
  author       = {Green, Michael and Ohlsson, Mattias},
  editor       = {Ifeachor, Emmanuel},
  keyword      = {performance estimation,k-fold cross validation,bootstrap,artificial neural networks},
  language     = {eng},
  pages        = {6},
  series       = {Third International Conference on Computational Intelligence in Medicine and Healthcare},
  title        = {Comparison of standard resampling methods for performance estimation of artificial neural network ensembles},
  year         = {2007},
}