Advanced

Clustering of back-end failures in automated testing

Folkesson, Fredrik LU and Nyholm, Johan (2016) In LU-CS-EX 2016-35 EDA920 20161
Department of Computer Science
Abstract (Swedish)
Automated Software testing is becoming increasingly popular, which in turn
creates more information that has to be analyzed. At the software company
Qlik a tool called NIOCAT is used to create clusters of failed test cases thought
to originate from the same code defect. The clustering is done in order to
decrease the ever increasing amount of manual analysis needed to be done with
regards to software testing. However, the existing tool currently only clusters
by using information from the front-end of the system under test. This makes
clusterings harder to create when the code defects which cause the tests to fail
are originating from the back-end.
In this thesis we have looked into different types of back-end information and
... (More)
Automated Software testing is becoming increasingly popular, which in turn
creates more information that has to be analyzed. At the software company
Qlik a tool called NIOCAT is used to create clusters of failed test cases thought
to originate from the same code defect. The clustering is done in order to
decrease the ever increasing amount of manual analysis needed to be done with
regards to software testing. However, the existing tool currently only clusters
by using information from the front-end of the system under test. This makes
clusterings harder to create when the code defects which cause the tests to fail
are originating from the back-end.
In this thesis we have looked into different types of back-end information and
different methods for using this information in order to create clusters of failed
test case executions originating from the same code defect. We created a prototype
that clusters failed test case executions by analyzing methods names
used in requests sent to the server. We did this using the vector space model
in which we evaluated multiple approaches for weighting terms. The best approach
seemed to be weighting the methods using a suspiciousness rating. The
prototype shows great promise of working well at Qlik but further work and
research has to be done to be conclusive. (Less)
Please use this url to cite or link to this publication:
author
Folkesson, Fredrik LU and Nyholm, Johan
supervisor
organization
course
EDA920 20161
year
type
H3 - Professional qualifications (4 Years - )
subject
keywords
clustering, test case, back-end, TF-IDF, suspicious statements, vector space model
publication/series
LU-CS-EX 2016-35
report number
LU-CS-EX 2016-35
ISSN
1650-2884
language
English
id
8889995
date added to LUP
2016-08-31 13:12:11
date last changed
2016-08-31 13:12:11
@misc{8889995,
  abstract     = {Automated Software testing is becoming increasingly popular, which in turn
creates more information that has to be analyzed. At the software company
Qlik a tool called NIOCAT is used to create clusters of failed test cases thought
to originate from the same code defect. The clustering is done in order to
decrease the ever increasing amount of manual analysis needed to be done with
regards to software testing. However, the existing tool currently only clusters
by using information from the front-end of the system under test. This makes
clusterings harder to create when the code defects which cause the tests to fail
are originating from the back-end.
In this thesis we have looked into different types of back-end information and
different methods for using this information in order to create clusters of failed
test case executions originating from the same code defect. We created a prototype
that clusters failed test case executions by analyzing methods names
used in requests sent to the server. We did this using the vector space model
in which we evaluated multiple approaches for weighting terms. The best approach
seemed to be weighting the methods using a suspiciousness rating. The
prototype shows great promise of working well at Qlik but further work and
research has to be done to be conclusive.},
  author       = {Folkesson, Fredrik and Nyholm, Johan},
  issn         = {1650-2884},
  keyword      = {clustering,test case,back-end,TF-IDF,suspicious statements,vector space model},
  language     = {eng},
  note         = {Student Paper},
  series       = {LU-CS-EX 2016-35},
  title        = {Clustering of back-end failures in automated testing},
  year         = {2016},
}