Unsupervised Anomaly Detection in Multivariate Time Series Using Variational Autoencoders
(2023) In Master's Theses in Mathematical Sciences FMAM05 20222Mathematics (Faculty of Engineering)
- Abstract
- In this master’s thesis, a novel unsupervised anomaly detection tool was developed in collaboration with Sandvik Rock Processing to assist engineers and experts in analyzing large amounts of sensor data from cone crushers used in the stone crushing industry. The tool focuses on analyzing power, pressure, and CSS sensor data. A crucial preprocessing step was implemented to algorithmically identify operation segments of sufficient length, differentiating between off, idle, continuous, and discontinuous states based on power usage.
The Variational Autoencoder (VAE) employed a unique architecture with two 1D convolutions in the encoder and 1D transposed convolution in the decoder, utilizing parallel kernel sizes of 2 and 15 to capture both... (More) - In this master’s thesis, a novel unsupervised anomaly detection tool was developed in collaboration with Sandvik Rock Processing to assist engineers and experts in analyzing large amounts of sensor data from cone crushers used in the stone crushing industry. The tool focuses on analyzing power, pressure, and CSS sensor data. A crucial preprocessing step was implemented to algorithmically identify operation segments of sufficient length, differentiating between off, idle, continuous, and discontinuous states based on power usage.
The Variational Autoencoder (VAE) employed a unique architecture with two 1D convolutions in the encoder and 1D transposed convolution in the decoder, utilizing parallel kernel sizes of 2 and 15 to capture both short-term and long-term patterns in the data. The decoder also incorporated a polynomial trend block to enhance the reconstruction. The VAE was trained on well-behaved operation segments to identify anomalous behaviour through the reconstruction error metric, Mean Absolute Percentage Error (MAPE). The anomaly detection tool achieved an F1 score of 0.89, 0.75, and 0.92 for the different sensors when tested with labelled anomalies provided by Sandvik.
Despite the challenge of limited labelled data, the tool successfully identifies the worst operation segments and can be utilized for deriving useful operation metrics. The main benefit of implementing this tool in the context of Sandvik Rock Processing’s operations is the significant acceleration of sensor data analysis and the ability to highlight areas of concern for engineers and experts. Potential future improvements include using a larger dataset for training, more rigorous testing of hyperparameters, and better data collection to account for factors such as machine models and expected operating pressure and power. (Less) - Popular Abstract
- Artificial Intelligence (AI) and Machine Learning (ML) are gaining popularity due to their ability to automate processes that otherwise require human intervention. The analysis of sensor data is a field that can gain immensely by the inclusion of AI methods. Carefully analysing weeks' worth of data is time-consuming and might be wasting an expert's time if most data is in regular operation. Using AI for data analysis can also help non-technical users form an analysis of machine behaviour.
This thesis project was done for Sandvik Rock Processing, aiming to develop a tool that can detect strange behaviour in the operation of stone crushers. The tool is meant to aid experts by alerting them to strange behaviour in large datasets. This is... (More) - Artificial Intelligence (AI) and Machine Learning (ML) are gaining popularity due to their ability to automate processes that otherwise require human intervention. The analysis of sensor data is a field that can gain immensely by the inclusion of AI methods. Carefully analysing weeks' worth of data is time-consuming and might be wasting an expert's time if most data is in regular operation. Using AI for data analysis can also help non-technical users form an analysis of machine behaviour.
This thesis project was done for Sandvik Rock Processing, aiming to develop a tool that can detect strange behaviour in the operation of stone crushers. The tool is meant to aid experts by alerting them to strange behaviour in large datasets. This is achieved by using an algorithm assessing power consumption patterns and automatically identifying and separating operational periods of different kinds, such as when the machine is off, idle, or in continuous or discontinuous use. The tool can then inform the user how often and how long the machine is in these four states.
When looking for strange behaviour segments of continuous operation are of interest. Here, a type of AI called a variational autoencoder (VAE) detects the strange operation segments. VAE comprises two deep neural networks (DNN): the encoder and decoder. The encoder shrinks the representation of the incoming data, and the decoder attempts to reconstruct the data from the shrunken representation. The encoder and decoder are trained together since the VAE learns to recreate the data. For the VAE to do this, it must make assumptions about the data's general behaviours. When the VAE is trained on exclusively well-behaving machine data, it becomes worse at reconstructing strangely behaving data. This is because now the data starts to violate some of those assumptions most likely accurate about well-behaved data. The error between the actual data and the reconstruction from the VAE can, therefore, be used as an indicator of strange behaviour. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9135876
- author
- Aronsson, Elias LU
- supervisor
- organization
- course
- FMAM05 20222
- year
- 2023
- type
- H1 - Master's Degree (One Year)
- subject
- keywords
- Anomaly Detection, Variational Autoencoder (VAE), Unsupervised Learning, Machine learning, AI
- publication/series
- Master's Theses in Mathematical Sciences
- report number
- LUTFMA-3517-2023
- ISSN
- 1404-6342
- other publication id
- 2023:E50
- language
- English
- id
- 9135876
- date added to LUP
- 2023-09-04 14:15:56
- date last changed
- 2023-09-04 14:15:56
@misc{9135876,
abstract = {{In this master’s thesis, a novel unsupervised anomaly detection tool was developed in collaboration with Sandvik Rock Processing to assist engineers and experts in analyzing large amounts of sensor data from cone crushers used in the stone crushing industry. The tool focuses on analyzing power, pressure, and CSS sensor data. A crucial preprocessing step was implemented to algorithmically identify operation segments of sufficient length, differentiating between off, idle, continuous, and discontinuous states based on power usage.
The Variational Autoencoder (VAE) employed a unique architecture with two 1D convolutions in the encoder and 1D transposed convolution in the decoder, utilizing parallel kernel sizes of 2 and 15 to capture both short-term and long-term patterns in the data. The decoder also incorporated a polynomial trend block to enhance the reconstruction. The VAE was trained on well-behaved operation segments to identify anomalous behaviour through the reconstruction error metric, Mean Absolute Percentage Error (MAPE). The anomaly detection tool achieved an F1 score of 0.89, 0.75, and 0.92 for the different sensors when tested with labelled anomalies provided by Sandvik.
Despite the challenge of limited labelled data, the tool successfully identifies the worst operation segments and can be utilized for deriving useful operation metrics. The main benefit of implementing this tool in the context of Sandvik Rock Processing’s operations is the significant acceleration of sensor data analysis and the ability to highlight areas of concern for engineers and experts. Potential future improvements include using a larger dataset for training, more rigorous testing of hyperparameters, and better data collection to account for factors such as machine models and expected operating pressure and power.}},
author = {{Aronsson, Elias}},
issn = {{1404-6342}},
language = {{eng}},
note = {{Student Paper}},
series = {{Master's Theses in Mathematical Sciences}},
title = {{Unsupervised Anomaly Detection in Multivariate Time Series Using Variational Autoencoders}},
year = {{2023}},
}