Integrated Error Detection at Software Launches with the Use of Machine Learning
(2024) EITM01 20241Department of Electrical and Information Technology
- Abstract
- During installation of firmware updates over the air, servers will usually collect data with statuscodes and information about the installation and update process. Given a large dataset with multiple installation steps, errors might occur occasionally dependant on outside factors such as internet connection, limited storage space or other issues that are not directly related to the stability of the update. Some updates however might have more issues in certain areas of the update process compared to the average. This thesis introduces an integrated system that detects anomalies on specific updates to determine the software stability of a released update. The system utilizes the unsupervised machine learning model isolation forest on device... (More)
- During installation of firmware updates over the air, servers will usually collect data with statuscodes and information about the installation and update process. Given a large dataset with multiple installation steps, errors might occur occasionally dependant on outside factors such as internet connection, limited storage space or other issues that are not directly related to the stability of the update. Some updates however might have more issues in certain areas of the update process compared to the average. This thesis introduces an integrated system that detects anomalies on specific updates to determine the software stability of a released update. The system utilizes the unsupervised machine learning model isolation forest on device aggregated data to highlight anomaly devices and make determinations about an update’s software stability. Using the isolation forest model with device aggregated data a high accuracy, recall, and precision metric is achieved while computing within a reasonable time. (Less)
- Popular Abstract
- As the world becomes more digitized, an increase in smart features is
needed both in mobile phones but also in other units all around us, like
alarms and sensors. Often companies want to make sure their product
has the latest feature and aims to release new updates to their products
over the air. However in doing so it is hard to make sure that everything
went right with the update and that no fault was introduced to the unit.
Sony has developed a cloud service that retrieves data events from Xperia
units for storage and calculation of different statistics from the data. The data events received by the cloud service are, amongst other information, status events for firmware updates conducted by the Xperia units. These events report... (More) - As the world becomes more digitized, an increase in smart features is
needed both in mobile phones but also in other units all around us, like
alarms and sensors. Often companies want to make sure their product
has the latest feature and aims to release new updates to their products
over the air. However in doing so it is hard to make sure that everything
went right with the update and that no fault was introduced to the unit.
Sony has developed a cloud service that retrieves data events from Xperia
units for storage and calculation of different statistics from the data. The data events received by the cloud service are, amongst other information, status events for firmware updates conducted by the Xperia units. These events report when a device has acknowledged an update, initiated/completed the download, verified the download and initiated/completed the installations, however the data events
also send errors when one or more of these processes goes wrong.
An assumption would be that the new firmware update is unstable if error
messages are sent to the cloud service, yet there are other outside factors that can result in error events that are quite common. A lost internet connection or not enough storage space on the device are two examples of innocent circumstances
that can result in these errors. If the firmware however is unstable it would be
useful to differentiate a normal sequence of firmware update events with that of a harmful firmware update.
To help differentiate a normal scenario with an unstable update scenario, ma-
chine learning can be used. Machine learning is a form of computer intelligence
that can be implemented in different ways depending on what problem needs to
be solved. In this project, update processes are compared to each other in order
to detect anomaly updates.
The final result of the project is an added feature to the cloud service that
on request can give a determination if a certain firmware release is stable or not depending on relevant units behavior during updates. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9164365
- author
- Westergården, Jakob LU and Vilhelmsson, Karl
- supervisor
- organization
- course
- EITM01 20241
- year
- 2024
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- Machine learning, data manipulation, Isolation Forest, ML-ops, software stability
- report number
- LU/LTH-EIT 2024-1003
- language
- English
- id
- 9164365
- date added to LUP
- 2024-06-26 14:54:01
- date last changed
- 2024-06-26 14:54:01
@misc{9164365,
abstract = {{During installation of firmware updates over the air, servers will usually collect data with statuscodes and information about the installation and update process. Given a large dataset with multiple installation steps, errors might occur occasionally dependant on outside factors such as internet connection, limited storage space or other issues that are not directly related to the stability of the update. Some updates however might have more issues in certain areas of the update process compared to the average. This thesis introduces an integrated system that detects anomalies on specific updates to determine the software stability of a released update. The system utilizes the unsupervised machine learning model isolation forest on device aggregated data to highlight anomaly devices and make determinations about an update’s software stability. Using the isolation forest model with device aggregated data a high accuracy, recall, and precision metric is achieved while computing within a reasonable time.}},
author = {{Westergården, Jakob and Vilhelmsson, Karl}},
language = {{eng}},
note = {{Student Paper}},
title = {{Integrated Error Detection at Software Launches with the Use of Machine Learning}},
year = {{2024}},
}