Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

An Empirically Grounded Path Forward for Scenario-Based Testing of Autonomous Driving Systems

Song, Qunying LU orcid ; Engström, Emelie LU orcid and Runeson, Per LU orcid (2024) 32nd ACM International Conference on the Foundations of Software Engineering, FSE Companion In FSE Companion - Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering p.232-243
Abstract

Testing of autonomous driving systems (ADS) is a crucial, yet complex task that requires different approaches to ensure the safety and reliability of the system in various driving scenarios. Currently, there is a lack of understanding of the industry practices for testing such systems, and also the related challenges. To this end, we conduct a secondary analysis of our previous exploratory study, where we interviewed 13 experts from 7 ADS companies in Sweden. We explore testing practices and challenges in industry, with a special focus on scenario-based testing as it is widely used in research for testing ADS. Through a detailed analysis and synthesis of the interviews, we identify key practices and challenges of testing ADS. Our... (More)

Testing of autonomous driving systems (ADS) is a crucial, yet complex task that requires different approaches to ensure the safety and reliability of the system in various driving scenarios. Currently, there is a lack of understanding of the industry practices for testing such systems, and also the related challenges. To this end, we conduct a secondary analysis of our previous exploratory study, where we interviewed 13 experts from 7 ADS companies in Sweden. We explore testing practices and challenges in industry, with a special focus on scenario-based testing as it is widely used in research for testing ADS. Through a detailed analysis and synthesis of the interviews, we identify key practices and challenges of testing ADS. Our analysis shows that the industry practices are primarily concerned with various types of testing methodologies, testing principles, selection and identification of test scenarios, test analysis, and relevant standards and tools as well as some general initiatives. Challenges mainly include discrepancies in concepts and methodologies used by different companies, together with a lack of comprehensive standards, regulations, and effective tools, approaches, and techniques for optimal testing. To address these issues, we propose a ‘3CO’ strategy (Combine, Collaborate, Continuously learn, and be Open) as a collective path forward for industry and academia to improve the testing frameworks for ADS.

(Less)
Please use this url to cite or link to this publication:
author
; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
autonomous driving systems, challenges, industry practices, interviews, scenario-based testing, software testing
host publication
FSE Companion - Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering
series title
FSE Companion - Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering
editor
d�Amorim, Marcelo
pages
12 pages
publisher
Association for Computing Machinery (ACM)
conference name
32nd ACM International Conference on the Foundations of Software Engineering, FSE Companion
conference location
Porto de Galinhas, Brazil
conference dates
2024-07-15 - 2024-07-19
external identifiers
  • scopus:85199111843
ISBN
9798400706585
DOI
10.1145/3663529.3663843
language
English
LU publication?
yes
additional info
Publisher Copyright: © 2024 Copyright held by the owner/author(s).
id
ad57d7bb-9d99-4ba8-9344-7fe004650684
date added to LUP
2024-07-31 14:09:04
date last changed
2024-09-19 12:04:56
@inproceedings{ad57d7bb-9d99-4ba8-9344-7fe004650684,
  abstract     = {{<p>Testing of autonomous driving systems (ADS) is a crucial, yet complex task that requires different approaches to ensure the safety and reliability of the system in various driving scenarios. Currently, there is a lack of understanding of the industry practices for testing such systems, and also the related challenges. To this end, we conduct a secondary analysis of our previous exploratory study, where we interviewed 13 experts from 7 ADS companies in Sweden. We explore testing practices and challenges in industry, with a special focus on scenario-based testing as it is widely used in research for testing ADS. Through a detailed analysis and synthesis of the interviews, we identify key practices and challenges of testing ADS. Our analysis shows that the industry practices are primarily concerned with various types of testing methodologies, testing principles, selection and identification of test scenarios, test analysis, and relevant standards and tools as well as some general initiatives. Challenges mainly include discrepancies in concepts and methodologies used by different companies, together with a lack of comprehensive standards, regulations, and effective tools, approaches, and techniques for optimal testing. To address these issues, we propose a ‘3CO’ strategy (Combine, Collaborate, Continuously learn, and be Open) as a collective path forward for industry and academia to improve the testing frameworks for ADS.</p>}},
  author       = {{Song, Qunying and Engström, Emelie and Runeson, Per}},
  booktitle    = {{FSE Companion - Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering}},
  editor       = {{d�Amorim, Marcelo}},
  isbn         = {{9798400706585}},
  keywords     = {{autonomous driving systems; challenges; industry practices; interviews; scenario-based testing; software testing}},
  language     = {{eng}},
  month        = {{07}},
  pages        = {{232--243}},
  publisher    = {{Association for Computing Machinery (ACM)}},
  series       = {{FSE Companion - Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering}},
  title        = {{An Empirically Grounded Path Forward for Scenario-Based Testing of Autonomous Driving Systems}},
  url          = {{http://dx.doi.org/10.1145/3663529.3663843}},
  doi          = {{10.1145/3663529.3663843}},
  year         = {{2024}},
}