Ensemble based unsupervised anomaly detection
(2017) EITM01 20162Department of Electrical and Information Technology
- Abstract
- A methodology as well as a suggested solution to the problem of unsupervised anomaly detection for contextual anomalies is presented. Using a combination of statistical and clustering approaches, an ensemble of algorithms provide automatic anomaly detection in an Application-to-person networking environment which can be scaled to different domains using hierarchical time series data.
The aim of this thesis is to further advance the field of anomaly detection and to provide conclusions with regards to the usability, maintainability and trustworthiness of unsupervised anomaly detection frameworks. Applications in the domain of unsupervised anomaly detection are hard to evaluate, thus methods as well as future work, which can be used to... (More) - A methodology as well as a suggested solution to the problem of unsupervised anomaly detection for contextual anomalies is presented. Using a combination of statistical and clustering approaches, an ensemble of algorithms provide automatic anomaly detection in an Application-to-person networking environment which can be scaled to different domains using hierarchical time series data.
The aim of this thesis is to further advance the field of anomaly detection and to provide conclusions with regards to the usability, maintainability and trustworthiness of unsupervised anomaly detection frameworks. Applications in the domain of unsupervised anomaly detection are hard to evaluate, thus methods as well as future work, which can be used to further create unmitigated assertions about any data set, is investigated.
An introduction to the concepts underlying anomaly detection as well as an implementation of the concepts are presented. Principles of machine learning are applied using static thresholds and assumptions about the data set being monitored. No active learning or dynamic adjustments of the anomaly detection framework is applied with the drawback of limiting the resulting classification but still providing clear and robust insights into the analyzed data.
It is shown that purely statistical or naive probabilistic assumptions about any data monitored is inconclusive in producing a fair estimation of anomalies. For a setting where the utility of an anomaly detection framework are not adamant to the survival of a monitoring system, the proposed solution works adequately. Since the results have not been validated, no conclusions can be drawn with regards to recall and precision metrics. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/8904850
- author
- Pieta Theofanous, Alexander LU and Alstersjö, Erik LU
- supervisor
-
- Zhi Zhang LU
- organization
- alternative title
- Sammansatta och oövervakade system för upptäckande av oregelbundenheter
- course
- EITM01 20162
- year
- 2017
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- unsupervised, ensemble based, computer engineering, Anomaly detection
- report number
- LU/LTH-EIT 2017-567
- language
- English
- id
- 8904850
- date added to LUP
- 2017-06-15 13:03:13
- date last changed
- 2017-06-15 13:03:13
@misc{8904850,
abstract = {{A methodology as well as a suggested solution to the problem of unsupervised anomaly detection for contextual anomalies is presented. Using a combination of statistical and clustering approaches, an ensemble of algorithms provide automatic anomaly detection in an Application-to-person networking environment which can be scaled to different domains using hierarchical time series data.
The aim of this thesis is to further advance the field of anomaly detection and to provide conclusions with regards to the usability, maintainability and trustworthiness of unsupervised anomaly detection frameworks. Applications in the domain of unsupervised anomaly detection are hard to evaluate, thus methods as well as future work, which can be used to further create unmitigated assertions about any data set, is investigated.
An introduction to the concepts underlying anomaly detection as well as an implementation of the concepts are presented. Principles of machine learning are applied using static thresholds and assumptions about the data set being monitored. No active learning or dynamic adjustments of the anomaly detection framework is applied with the drawback of limiting the resulting classification but still providing clear and robust insights into the analyzed data.
It is shown that purely statistical or naive probabilistic assumptions about any data monitored is inconclusive in producing a fair estimation of anomalies. For a setting where the utility of an anomaly detection framework are not adamant to the survival of a monitoring system, the proposed solution works adequately. Since the results have not been validated, no conclusions can be drawn with regards to recall and precision metrics.}},
author = {{Pieta Theofanous, Alexander and Alstersjö, Erik}},
language = {{eng}},
note = {{Student Paper}},
title = {{Ensemble based unsupervised anomaly detection}},
year = {{2017}},
}