Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Testing Abstractions for Cyber-Physical Control Systems

Mandrioli, Claudio LU orcid ; Nyberg Carlsson, Max LU orcid and Maggio, Martina LU (2023) In ACM Transactions on Software Engineering and Methodology
Abstract
Control systems are ubiquitous and often at the core of Cyber-Physical Systems, like cars and aeroplanes. They are implemented as embedded software, that interacts in closed loop with the physical world through sensors and actuators. As a consequence, the software cannot just be tested in isolation. To close the loop in a testing environment and root causing failure generated by different parts of the system, executable models are used to abstract specific components. Different testing setups can be implemented by abstracting different elements: the most common ones are model-in-the-loop, software-in-the-loop, hardware-in-the-loop, and real-physics-in-the-loop. In this paper, we discuss the properties of these setups and the types of... (More)
Control systems are ubiquitous and often at the core of Cyber-Physical Systems, like cars and aeroplanes. They are implemented as embedded software, that interacts in closed loop with the physical world through sensors and actuators. As a consequence, the software cannot just be tested in isolation. To close the loop in a testing environment and root causing failure generated by different parts of the system, executable models are used to abstract specific components. Different testing setups can be implemented by abstracting different elements: the most common ones are model-in-the-loop, software-in-the-loop, hardware-in-the-loop, and real-physics-in-the-loop. In this paper, we discuss the properties of these setups and the types of faults they can expose. We develop a comprehensive case study using the Crazyflie, a drone whose software and hardware are open-source. We implement all the most common testing setups and ensure the consistent injection of faults in each of them. We inject faults in the control system and we compare with the nominal performance of the non-faulty software. Our results show the specific capabilities of the different setups in exposing faults. Contrary to intuition and previous literature, we show that the setups do not belong to a strict hierarchy and they are best designed to maximize the differences across them rather than to be as close as possible to reality. (Less)
Please use this url to cite or link to this publication:
author
; and
organization
publishing date
type
Contribution to journal
publication status
published
subject
in
ACM Transactions on Software Engineering and Methodology
publisher
Association for Computing Machinery (ACM)
external identifiers
  • scopus:85180262411
ISSN
1049-331X
DOI
10.1145/3617170
language
English
LU publication?
yes
id
00a11a70-a3e1-4a86-87e8-8be9f4ca9671
date added to LUP
2023-10-31 10:29:02
date last changed
2024-03-29 04:01:09
@article{00a11a70-a3e1-4a86-87e8-8be9f4ca9671,
  abstract     = {{Control systems are ubiquitous and often at the core of Cyber-Physical Systems, like cars and aeroplanes. They are implemented as embedded software, that interacts in closed loop with the physical world through sensors and actuators. As a consequence, the software cannot just be tested in isolation. To close the loop in a testing environment and root causing failure generated by different parts of the system, executable models are used to abstract specific components. Different testing setups can be implemented by abstracting different elements: the most common ones are model-in-the-loop, software-in-the-loop, hardware-in-the-loop, and real-physics-in-the-loop. In this paper, we discuss the properties of these setups and the types of faults they can expose. We develop a comprehensive case study using the Crazyflie, a drone whose software and hardware are open-source. We implement all the most common testing setups and ensure the consistent injection of faults in each of them. We inject faults in the control system and we compare with the nominal performance of the non-faulty software. Our results show the specific capabilities of the different setups in exposing faults. Contrary to intuition and previous literature, we show that the setups do not belong to a strict hierarchy and they are best designed to maximize the differences across them rather than to be as close as possible to reality.}},
  author       = {{Mandrioli, Claudio and Nyberg Carlsson, Max and Maggio, Martina}},
  issn         = {{1049-331X}},
  language     = {{eng}},
  publisher    = {{Association for Computing Machinery (ACM)}},
  series       = {{ACM Transactions on Software Engineering and Methodology}},
  title        = {{Testing Abstractions for Cyber-Physical Control Systems}},
  url          = {{http://dx.doi.org/10.1145/3617170}},
  doi          = {{10.1145/3617170}},
  year         = {{2023}},
}