Continuous Validation of Multi-Cloud Systems by Automated Test Scripting
(2019) EITM01 20191Department of Electrical and Information Technology
- Abstract
- Scalability, pay-as-you-go, and flexibility are some factors why more and more organizations are utilizing a cloud environment for their IT systems. Specifically, the trend is towards multi-cloud, where different cloud providers manage different applications and services. This environment is however a lot less deterministic than an on premise solution. For example, if an application from one cloud provider is updated, this might result in that another application from another cloud provider stops working. This Master’s thesis investigates how a system owner can validate his/her cloud-based business system continuously, to be able to see that it functions as expected, as well as to spot anomalies. A measurement method is developed using... (More)
- Scalability, pay-as-you-go, and flexibility are some factors why more and more organizations are utilizing a cloud environment for their IT systems. Specifically, the trend is towards multi-cloud, where different cloud providers manage different applications and services. This environment is however a lot less deterministic than an on premise solution. For example, if an application from one cloud provider is updated, this might result in that another application from another cloud provider stops working. This Master’s thesis investigates how a system owner can validate his/her cloud-based business system continuously, to be able to see that it functions as expected, as well as to spot anomalies. A measurement method is developed using Apache JMeter and crontab, with a setup using an Amazon EC2 instance and Raspberry Pis. To test this method, a multiple case study is performed on two different companies with two different cloud environments. Three tests are performed. One measures the login process, among other things, another measures the time required to generate a report of inventory values, and the third measures the response times of an ordering system. The method is successful for both environments, and all tests, as is shown by this thesis. For the first and third test, the response times are visualized in graphs. The result of the second test is shown in a table. The two companies are different in the sense that one is a large international company, while the other is a medium-sized company just located in Sweden. Also, the third test is performed because this company experienced its ordering system as slow, while the other company had no reported anomalies. Hence, this measurement method can be used to test different functions in a business system for both large and medium-sized companies. Furthermore, if a company suspects an error in their system, this method can be used to verify its existence, as well as verify if the error has been solved or not. (Less)
- Popular Abstract
- It has become very popular for organizations to move their IT systems to the cloud. However, this environment can be quite unpredictable and changes constantly. How does this affect an organization's cloud-based IT system? Well, the answer is to measure, to know.
----------------------------------------------------------------------
Have you ever used Google's email service? Perhaps you have a few documents saved in Google Drive or use Dropbox as an additional backup for your photos? You access them through the Internet without having to maintain or support them yourself. This is because they are all applications based in the cloud.
Cloud-based applications and services are maintained by cloud providers, such as Amazon, Microsoft,... (More) - It has become very popular for organizations to move their IT systems to the cloud. However, this environment can be quite unpredictable and changes constantly. How does this affect an organization's cloud-based IT system? Well, the answer is to measure, to know.
----------------------------------------------------------------------
Have you ever used Google's email service? Perhaps you have a few documents saved in Google Drive or use Dropbox as an additional backup for your photos? You access them through the Internet without having to maintain or support them yourself. This is because they are all applications based in the cloud.
Cloud-based applications and services are maintained by cloud providers, such as Amazon, Microsoft, and Google. If an organization would use multiple cloud providers for their IT system, for example the storage service from Amazon and the analyzing tools from Microsoft, they are utilizing a multi-cloud environment. The providers are independent of each other. Thus, an IT system in this environment can stop working as expected if, for example, one provider adds a new version to its service, which is incompatible with the other providers' services. To be able to see how new changes affect a system, validation can be performed. Also, since the cloud changes constantly, this can with advantage be performed continuously.
This Master's thesis investigates how continuous validation can be executed, and a measurement method is developed. By performing measurements on a system continuously, this method can clarify what normal behaviour is and what is not. It is used for gaining knowledge about a system, for example measuring the time it takes to log in. It can also be used when trying to solve an error. For example, if users complain about a system being slow, this method can tell if the system has regained its speed again by measuring both before and after the potential solution took place.
A multiple case study is performed to evaluate the method, and for both test cases included, the method is considered successful. The method includes a lot of manual work, which might be a drawback for some organisations. However, this is considered to have less impact, comparing with the overall advantages with continuous validation. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/8985751
- author
- Sandström, Josefine LU
- supervisor
- organization
- course
- EITM01 20191
- year
- 2019
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- Continuous validation, Multi-cloud
- report number
- LU/LTH-EIT 2019-704
- language
- English
- id
- 8985751
- date added to LUP
- 2019-06-20 10:38:26
- date last changed
- 2019-06-20 10:38:26
@misc{8985751,
abstract = {{Scalability, pay-as-you-go, and flexibility are some factors why more and more organizations are utilizing a cloud environment for their IT systems. Specifically, the trend is towards multi-cloud, where different cloud providers manage different applications and services. This environment is however a lot less deterministic than an on premise solution. For example, if an application from one cloud provider is updated, this might result in that another application from another cloud provider stops working. This Master’s thesis investigates how a system owner can validate his/her cloud-based business system continuously, to be able to see that it functions as expected, as well as to spot anomalies. A measurement method is developed using Apache JMeter and crontab, with a setup using an Amazon EC2 instance and Raspberry Pis. To test this method, a multiple case study is performed on two different companies with two different cloud environments. Three tests are performed. One measures the login process, among other things, another measures the time required to generate a report of inventory values, and the third measures the response times of an ordering system. The method is successful for both environments, and all tests, as is shown by this thesis. For the first and third test, the response times are visualized in graphs. The result of the second test is shown in a table. The two companies are different in the sense that one is a large international company, while the other is a medium-sized company just located in Sweden. Also, the third test is performed because this company experienced its ordering system as slow, while the other company had no reported anomalies. Hence, this measurement method can be used to test different functions in a business system for both large and medium-sized companies. Furthermore, if a company suspects an error in their system, this method can be used to verify its existence, as well as verify if the error has been solved or not.}},
author = {{Sandström, Josefine}},
language = {{eng}},
note = {{Student Paper}},
title = {{Continuous Validation of Multi-Cloud Systems by Automated Test Scripting}},
year = {{2019}},
}