Mutation testing optimisations using the Clang front-end
Vercammen, Sten LU ; Demeyer, Serge LU ; Borg, Markus LU ; Pettersson, Niklas and Hedin, Görel LU
(2024)
In Software Testing Verification and Reliability
34(1).
- Abstract
Mutation testing is the state-of-the-art technique for assessing the fault detection capacity of a test suite. Unfortunately, a full mutation analysis is often prohibitively expensive. The CppCheck project for instance, demands a build time of 5.8 min and a test execution time of 17 s on our desktop computer. An unoptimised mutation analysis, for 55,000 generated mutants took 11.8 days in total, of which 4.3 days is spent on (re)compiling the project. In this paper, we present a feasibility study, investigating how a number of optimisation strategies can be implemented based on the Clang front-end. These optimisation strategies allow to eliminate the compilation and execution overhead in order to support efficient mutation testing for... (More)
Mutation testing is the state-of-the-art technique for assessing the fault detection capacity of a test suite. Unfortunately, a full mutation analysis is often prohibitively expensive. The CppCheck project for instance, demands a build time of 5.8 min and a test execution time of 17 s on our desktop computer. An unoptimised mutation analysis, for 55,000 generated mutants took 11.8 days in total, of which 4.3 days is spent on (re)compiling the project. In this paper, we present a feasibility study, investigating how a number of optimisation strategies can be implemented based on the Clang front-end. These optimisation strategies allow to eliminate the compilation and execution overhead in order to support efficient mutation testing for the C language family. We provide a proof-of-concept tool that achieves a speedup of between 2 (Formula presented.) and 30 (Formula presented.). We make a detailed analysis of the speedup induced by the optimisations, elaborate on the lessons learned and point out avenues for further improvements.
(Less)
- author
- Vercammen, Sten
LU
; Demeyer, Serge
LU
; Borg, Markus
LU
; Pettersson, Niklas
and Hedin, Görel
LU
- organization
- publishing date
- 2024
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- C++, CLANG, mutant schemata, mutation testing
- in
- Software Testing Verification and Reliability
- volume
- 34
- issue
- 1
- publisher
- John Wiley & Sons Inc.
- external identifiers
-
- scopus:85174318138
- ISSN
- 0960-0833
- DOI
- 10.1002/stvr.1865
- language
- English
- LU publication?
- yes
- additional info
- Funding Information: This work is supported by (a) the Research Foundation Flanders (FWO) under Grant number 1SA1519N; (a) the FWO‐Vlaanderen and F.R.S.‐FNRS via the Excellence of Science project 30446992 SECO‐ASSIST. Publisher Copyright: © 2023 The Authors. Software Testing, Verification & Reliability published by John Wiley & Sons Ltd.
- id
- d82edfca-0236-4c94-a798-f919e7296b2d
- date added to LUP
- 2023-12-13 14:03:14
- date last changed
- 2024-02-09 11:56:32
@article{d82edfca-0236-4c94-a798-f919e7296b2d,
abstract = {{<p>Mutation testing is the state-of-the-art technique for assessing the fault detection capacity of a test suite. Unfortunately, a full mutation analysis is often prohibitively expensive. The CppCheck project for instance, demands a build time of 5.8 min and a test execution time of 17 s on our desktop computer. An unoptimised mutation analysis, for 55,000 generated mutants took 11.8 days in total, of which 4.3 days is spent on (re)compiling the project. In this paper, we present a feasibility study, investigating how a number of optimisation strategies can be implemented based on the Clang front-end. These optimisation strategies allow to eliminate the compilation and execution overhead in order to support efficient mutation testing for the C language family. We provide a proof-of-concept tool that achieves a speedup of between 2 (Formula presented.) and 30 (Formula presented.). We make a detailed analysis of the speedup induced by the optimisations, elaborate on the lessons learned and point out avenues for further improvements.</p>}},
author = {{Vercammen, Sten and Demeyer, Serge and Borg, Markus and Pettersson, Niklas and Hedin, Görel}},
issn = {{0960-0833}},
keywords = {{C++; CLANG; mutant schemata; mutation testing}},
language = {{eng}},
number = {{1}},
publisher = {{John Wiley & Sons Inc.}},
series = {{Software Testing Verification and Reliability}},
title = {{Mutation testing optimisations using the Clang front-end}},
url = {{http://dx.doi.org/10.1002/stvr.1865}},
doi = {{10.1002/stvr.1865}},
volume = {{34}},
year = {{2024}},
}