Dynamic Stopping for Artificial Neural Networks
(2022) FYTK02 20212Computational Biology and Biological Physics - Undergoing reorganization
- Abstract
- The growing popularity of Artificial Neural Networks (ANN) demands continuous improvement and optimization of the training process to achieve higher-performing algorithms at a lower computational cost. During training an ANN learns to solve a problem by looking at examples, and will iteratively go over a dataset to reach an optimal performance. Usually the user needs to define a fixed number of iterations before viewing the final result and assessing the quality of training. The goal of the project presented in this paper was to optimize the training process by defining an automatic stopping criterion, which arrests training when good performance is achieved but further training would yield diminishing returns. The two methods explored to... (More)
- The growing popularity of Artificial Neural Networks (ANN) demands continuous improvement and optimization of the training process to achieve higher-performing algorithms at a lower computational cost. During training an ANN learns to solve a problem by looking at examples, and will iteratively go over a dataset to reach an optimal performance. Usually the user needs to define a fixed number of iterations before viewing the final result and assessing the quality of training. The goal of the project presented in this paper was to optimize the training process by defining an automatic stopping criterion, which arrests training when good performance is achieved but further training would yield diminishing returns. The two methods explored to achieve this were based on the Plateau Detection coefficient (C) and the Gradient Correlation coefficient (G). The hypothesis was that these could be used across different problems and different networks with consistently good dynamic stopping. Testing over multiple different ANN parameters revealed that G=1 produced a good stopping criterion over almost all scenarios, whereas C presented some weaknesses. (Less)
- Popular Abstract
- “Artificial Intelligence” is a term that is now ubiquitous in media, entertainment, social media and even the engineering world. Most people are confused about what it is, where it is and what it’s made of. The term is even thrown around in philosophical debates, in which people attempt to give meaning to the abstract concepts of “artificial” and “intelligence”. So, I would like to introduce an alternative term for AI: the Artificial Neural Network (a.k.a. ANN). Using “Neural Network” is useful because it is more specific: we are talking about a network made of neurons. Just like the human brain, an ANN is made up of individual units (called neurons) which communicate with each other. When they are all working together, they can perform... (More)
- “Artificial Intelligence” is a term that is now ubiquitous in media, entertainment, social media and even the engineering world. Most people are confused about what it is, where it is and what it’s made of. The term is even thrown around in philosophical debates, in which people attempt to give meaning to the abstract concepts of “artificial” and “intelligence”. So, I would like to introduce an alternative term for AI: the Artificial Neural Network (a.k.a. ANN). Using “Neural Network” is useful because it is more specific: we are talking about a network made of neurons. Just like the human brain, an ANN is made up of individual units (called neurons) which communicate with each other. When they are all working together, they can perform complicated operations much faster than one node on its own.
The reason why ANNs are so fast and flexible is because they undergo a training stage, where the network "practices" on training points and in response "tunes" the values of its nodes, which affects the final output. This Bachelor Thesis takes a closer look
The second way that was studied is the Gradient Correlation coefficient. After a network trains on a number of data-points, it receives a feedback in the form of the gradient. The gradient contains information for how individual nodes and parameters need to be adjusted in the next training instance to improve the networks performance. By analysing the gradient we were able to define a coefficient that, when set to the value 1, indicated precise stopping points in training.
After studying how these methods behaved in different scenarios, it was shown that the Gradient Correlation method was better at predicting reliable stopping points than the Plateau Detection method, and that it would be worth it to explore it more. After all, this is only the beginning of the study of this coefficient, and if it turns out to be a reliable way of stopping training in ANNs, its widespread use could mean advantageous improvements in efficiency of these networks. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9078057
- author
- Lizotte, Daniel Louis Hubert LU
- supervisor
-
- Patrik Edén LU
- organization
- course
- FYTK02 20212
- year
- 2022
- type
- M2 - Bachelor Degree
- subject
- keywords
- Artificial Neural Network
- language
- English
- id
- 9078057
- date added to LUP
- 2022-04-08 09:39:40
- date last changed
- 2022-06-30 11:29:18
@misc{9078057,
abstract = {{The growing popularity of Artificial Neural Networks (ANN) demands continuous improvement and optimization of the training process to achieve higher-performing algorithms at a lower computational cost. During training an ANN learns to solve a problem by looking at examples, and will iteratively go over a dataset to reach an optimal performance. Usually the user needs to define a fixed number of iterations before viewing the final result and assessing the quality of training. The goal of the project presented in this paper was to optimize the training process by defining an automatic stopping criterion, which arrests training when good performance is achieved but further training would yield diminishing returns. The two methods explored to achieve this were based on the Plateau Detection coefficient (C) and the Gradient Correlation coefficient (G). The hypothesis was that these could be used across different problems and different networks with consistently good dynamic stopping. Testing over multiple different ANN parameters revealed that G=1 produced a good stopping criterion over almost all scenarios, whereas C presented some weaknesses.}},
author = {{Lizotte, Daniel Louis Hubert}},
language = {{eng}},
note = {{Student Paper}},
title = {{Dynamic Stopping for Artificial Neural Networks}},
year = {{2022}},
}