Combining Cross-Validation and Ensemble Creation for Artificial Neural Networks
(2022) FYTK02 20221Computational Biology and Biological Physics - Undergoing reorganization
- Abstract
- Artificial neural networks (ANNs) are widely used nowadays, and the research into improving their performances is continually ongoing. One main goal of ANNs is to have a high generalization performance, which can be estimated through validation. Ensembles can be useful to raise the generalization performance, but the validation of ensembles is often computationally costly if the size of the training data set is limited. Therefore, this thesis introduces shortcut ensembles during cross-validation, where several validation outputs get averaged to estimate the generalization performance of an ensemble. To evaluate this method the validation performance of the shortcut ensemble was compared to validation and test performances of a single model... (More)
- Artificial neural networks (ANNs) are widely used nowadays, and the research into improving their performances is continually ongoing. One main goal of ANNs is to have a high generalization performance, which can be estimated through validation. Ensembles can be useful to raise the generalization performance, but the validation of ensembles is often computationally costly if the size of the training data set is limited. Therefore, this thesis introduces shortcut ensembles during cross-validation, where several validation outputs get averaged to estimate the generalization performance of an ensemble. To evaluate this method the validation performance of the shortcut ensemble was compared to validation and test performances of a single model and an actual ensemble, using two different data sets for classification problems. The results show that the shortcut ensemble gives better estimates for the generalization performance of an ensemble than a single model during validation and it can approximate the validation performance of an actual ensemble. Hence, the shortcut ensemble can provide a less costly way of validating ensembles during cross-validation. (Less)
- Popular Abstract
- Robots are already playing a big role in our world, and if you dive into the world of science fiction, you will very quickly come across stories of robots with artificial intelligence taking over the world. Is this a realistic scenario? In reality, artificial intelligence is far away from human thinking. But research is constantly aiming to improve artificial neural networks - networks that partly mimic functions of human brains. This thesis investigates a method, so-called shortcut ensembles, that might have the potential to improve their performance at least a little bit.
A central function of our brain is to make conscious decisions, and everyone knows that making important decisions is hard and can take a bit of time. We all... (More) - Robots are already playing a big role in our world, and if you dive into the world of science fiction, you will very quickly come across stories of robots with artificial intelligence taking over the world. Is this a realistic scenario? In reality, artificial intelligence is far away from human thinking. But research is constantly aiming to improve artificial neural networks - networks that partly mimic functions of human brains. This thesis investigates a method, so-called shortcut ensembles, that might have the potential to improve their performance at least a little bit.
A central function of our brain is to make conscious decisions, and everyone knows that making important decisions is hard and can take a bit of time. We all experienced the situation where we had a decision that we liked to discuss with other people to collect their opinions before choosing ourselves. Usually, we feel that our decision has been approved if many people agree with it, and it is more likely that we made a good decision. Similarly, if you want an artificial neural network to make a decision it can be helpful to also ask more than one network for an opinion and then choose the decision that agrees with the most networks. This process is called ensemble learning and is already widely used.
In general, artificial brains need to be trained, similarly as we humans have to study if we want to increase our ability to make decisions. While training, there are several commonly used possibilities to check the performance of an artificial neural network. However, some of them are very costly in combination with ensemble learning, since then tedious computations need to be performed and lots of time is consumed. Here, the new method of creating shortcut ensembles could be able to help. The method suggests asking for opinions from other networks directly during the process of checking the performance.
After studying the shortcut ensembles, it was possible to conclude that they are indeed useful to approximate performances that can be reached when using ensemble learning with artificial neural networks. There is still more research needed, testing the new method on more complex systems, but it can make ensemble learning easier and more efficient. Furthermore, ensemble learning might become an even more used learning method, helping artificial brains to make better decisions. With this improvement, robots including artificial intelligence might not be able to rule the world, but they could give us greater help in everyday life. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9089542
- author
- Hölldobler, Anna Lena LU
- supervisor
-
- Patrik Edén LU
- organization
- course
- FYTK02 20221
- year
- 2022
- type
- M2 - Bachelor Degree
- subject
- keywords
- Artificial Neural Networks, Cross-Validation, Ensemble Creation
- language
- English
- id
- 9089542
- date added to LUP
- 2022-06-23 11:18:32
- date last changed
- 2022-06-29 15:20:48
@misc{9089542,
abstract = {{Artificial neural networks (ANNs) are widely used nowadays, and the research into improving their performances is continually ongoing. One main goal of ANNs is to have a high generalization performance, which can be estimated through validation. Ensembles can be useful to raise the generalization performance, but the validation of ensembles is often computationally costly if the size of the training data set is limited. Therefore, this thesis introduces shortcut ensembles during cross-validation, where several validation outputs get averaged to estimate the generalization performance of an ensemble. To evaluate this method the validation performance of the shortcut ensemble was compared to validation and test performances of a single model and an actual ensemble, using two different data sets for classification problems. The results show that the shortcut ensemble gives better estimates for the generalization performance of an ensemble than a single model during validation and it can approximate the validation performance of an actual ensemble. Hence, the shortcut ensemble can provide a less costly way of validating ensembles during cross-validation.}},
author = {{Hölldobler, Anna Lena}},
language = {{eng}},
note = {{Student Paper}},
title = {{Combining Cross-Validation and Ensemble Creation for Artificial Neural Networks}},
year = {{2022}},
}