Efficient evaluation of multifactor dependent system performance using fractional factorial design
Berling, Tomas LU and Runeson, Per LU
(2003)
In IEEE Transactions on Software Engineering
29(9).
p.769-781
- Abstract
- Performance of computer-based systems may depend on many different factors, internal and external. In order to design a system to have the desired performance or to validate that the system has the required performance, the effect of the influencing factors must be known. Common methods give no or little guidance on how to vary the factors during prototyping or validation. Varying the factors in all possible combinations would be too expensive and too time-consuming. This paper introduces a systematic approach to the prototyping and the validation of a system's performance, by treating the prototyping or validation as an experiment, in which the fractional factorial design methodology is commonly used. To show that this is possible, a case... (More)
- Performance of computer-based systems may depend on many different factors, internal and external. In order to design a system to have the desired performance or to validate that the system has the required performance, the effect of the influencing factors must be known. Common methods give no or little guidance on how to vary the factors during prototyping or validation. Varying the factors in all possible combinations would be too expensive and too time-consuming. This paper introduces a systematic approach to the prototyping and the validation of a system's performance, by treating the prototyping or validation as an experiment, in which the fractional factorial design methodology is commonly used. To show that this is possible, a case study evaluating the influencing factors of the false and real target rate of a radar system is described. Our findings show that prototyping and validation of system performance become structured and effective when using the fractional factorial design. The methodology enables planning, performance, structured analysis, and gives guidance for appropriate test cases. The methodology yields not only main factors, but also interacting factors. The effort is minimized for finding the results, due to the methodology. The case study shows that after 112 test Cases, of 1,024 possible, the knowledge gained was enough to draw conclusions on the effects and interactions of 10 factors. This is a reduction with a factor 5-9 compared to alternative methods. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/301736
- author
- Berling, Tomas
LU
and Runeson, Per
LU
- organization
- publishing date
- 2003
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- evaluation, performance of systems, fractional factorial design, prototyping, performance
- in
- IEEE Transactions on Software Engineering
- volume
- 29
- issue
- 9
- pages
- 769 - 781
- publisher
- IEEE - Institute of Electrical and Electronics Engineers Inc.
- external identifiers
-
- wos:000185191700001
- scopus:0142134961
- ISSN
- 0098-5589
- DOI
- 10.1109/TSE.2003.1232283
- language
- English
- LU publication?
- yes
- id
- d1422383-e155-48d7-a6f7-645bc8ae2903 (old id 301736)
- date added to LUP
- 2016-04-01 16:32:09
- date last changed
- 2022-01-28 20:23:11
@article{d1422383-e155-48d7-a6f7-645bc8ae2903,
abstract = {{Performance of computer-based systems may depend on many different factors, internal and external. In order to design a system to have the desired performance or to validate that the system has the required performance, the effect of the influencing factors must be known. Common methods give no or little guidance on how to vary the factors during prototyping or validation. Varying the factors in all possible combinations would be too expensive and too time-consuming. This paper introduces a systematic approach to the prototyping and the validation of a system's performance, by treating the prototyping or validation as an experiment, in which the fractional factorial design methodology is commonly used. To show that this is possible, a case study evaluating the influencing factors of the false and real target rate of a radar system is described. Our findings show that prototyping and validation of system performance become structured and effective when using the fractional factorial design. The methodology enables planning, performance, structured analysis, and gives guidance for appropriate test cases. The methodology yields not only main factors, but also interacting factors. The effort is minimized for finding the results, due to the methodology. The case study shows that after 112 test Cases, of 1,024 possible, the knowledge gained was enough to draw conclusions on the effects and interactions of 10 factors. This is a reduction with a factor 5-9 compared to alternative methods.}},
author = {{Berling, Tomas and Runeson, Per}},
issn = {{0098-5589}},
keywords = {{evaluation; performance of systems; fractional factorial design; prototyping; performance}},
language = {{eng}},
number = {{9}},
pages = {{769--781}},
publisher = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
series = {{IEEE Transactions on Software Engineering}},
title = {{Efficient evaluation of multifactor dependent system performance using fractional factorial design}},
url = {{http://dx.doi.org/10.1109/TSE.2003.1232283}},
doi = {{10.1109/TSE.2003.1232283}},
volume = {{29}},
year = {{2003}},
}