Critical Scenario Identification for Testing of Realistic Autonomous Driving Systems
Song, Qunying LU
(2024)
- Abstract
- Background: Testing is imperative to validate the functionalities and safety of autonomous driving systems. Simulated scenario-based testing is commonly adopted for autonomous driving systems, which aims to construct various driving scenarios and validate the autonomous driving systems in simulation. Nevertheless, identifying relevant test scenarios, especially critical ones that expose hazards or risks of harm to autonomous vehicles remains an open challenge.
Objective: We focus on critical scenario identification for testing of realistic autonomous driving systems in this thesis. Specifically, our objective is to establish an effective approach for identifying critical scenarios for testing of industrial autonomous driving... (More) - Background: Testing is imperative to validate the functionalities and safety of autonomous driving systems. Simulated scenario-based testing is commonly adopted for autonomous driving systems, which aims to construct various driving scenarios and validate the autonomous driving systems in simulation. Nevertheless, identifying relevant test scenarios, especially critical ones that expose hazards or risks of harm to autonomous vehicles remains an open challenge.
Objective: We focus on critical scenario identification for testing of realistic autonomous driving systems in this thesis. Specifically, our objective is to establish an effective approach for identifying critical scenarios for testing of industrial autonomous driving systems. Also, we aim to explore and improve current practices of testing autonomous driving systems with critical scenarios in industry.
Methodology: We follow the design science research paradigm and perform two iterations of the design research cycle in this thesis. The first iteration focuses on the first objective of the thesis to design an approach for critical scenario identification. In the second iteration, we explore industry practices of using critical scenarios for testing of autonomous driving systems, and propose a preliminary solution to evaluate the realism of such scenarios and improve their validity.
Contributions: The thesis comprises five studies and our contributions are summarized into four folds. (1) Taxonomies of concepts, challenges, practices, techniques, and approaches for testing autonomous systems. (2) A workbench with interconnected tools and a workflow for end-to-end critical scenario identification, which is demonstrated on two industrial autonomous driving systems. (3) Taxonomies of industry practices, initiatives, and challenges for identifying and using critical scenarios for testing autonomous driving systems. (4) A methodology to evaluate the realism of synthetic critical scenarios and a proof of concept of it on two autonomous vehicle collision scenario sets.
Conclusion: We explore the industry practices of and design an effective approach for critical scenario identification as well as a methodology to evaluate the realism of the resulting scenarios. The current practices of critical scenario identification are limited and hindered by several challenges such as cost, tool support, and scenario realism. Therefore, we recommend the industry and academia combine existing approaches, collaborate, and continuously learn to improve their practical use for testing of autonomous driving systems. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/fb1d922c-5d87-4dcd-bedc-1d47f8f1dd8d
- author
- Song, Qunying
LU
- supervisor
-
- Per Runeson LU
- Emelie Engström LU
- opponent
-
- Prof. Pfahl, Dietmar, University of Tartu, Estonia.
- organization
- publishing date
- 2024
- type
- Thesis
- publication status
- published
- subject
- keywords
- Autonomous driving systems, software testing, Search-based testing, Simulation, Critical scenario, Critical scenario identification, Industry practices, Challenges, Realism
- pages
- 206 pages
- publisher
- Computer Science, Lund University
- defense location
- Lecture Hall E:1406, building E, Ole Römers väg 3, Faculty of Engineering LTH, Lund University, Lund. The dissertation will be live streamed, but part of the premises is to be excluded from the live stream. Zoom: https://lu-se.zoom.us/j/61663216784
- defense date
- 2024-11-15 09:15:00
- ISBN
- 978-91-8104-243-6
- 978-91-8104-244-3
- language
- English
- LU publication?
- yes
- id
- fb1d922c-5d87-4dcd-bedc-1d47f8f1dd8d
- date added to LUP
- 2024-10-16 09:52:57
- date last changed
- 2025-04-04 13:53:48
@phdthesis{fb1d922c-5d87-4dcd-bedc-1d47f8f1dd8d,
abstract = {{Background: Testing is imperative to validate the functionalities and safety of autonomous driving systems. Simulated scenario-based testing is commonly adopted for autonomous driving systems, which aims to construct various driving scenarios and validate the autonomous driving systems in simulation. Nevertheless, identifying relevant test scenarios, especially critical ones that expose hazards or risks of harm to autonomous vehicles remains an open challenge.<br/><br/>Objective: We focus on critical scenario identification for testing of realistic autonomous driving systems in this thesis. Specifically, our objective is to establish an effective approach for identifying critical scenarios for testing of industrial autonomous driving systems. Also, we aim to explore and improve current practices of testing autonomous driving systems with critical scenarios in industry. <br/><br/>Methodology: We follow the design science research paradigm and perform two iterations of the design research cycle in this thesis. The first iteration focuses on the first objective of the thesis to design an approach for critical scenario identification. In the second iteration, we explore industry practices of using critical scenarios for testing of autonomous driving systems, and propose a preliminary solution to evaluate the realism of such scenarios and improve their validity.<br/><br/>Contributions: The thesis comprises five studies and our contributions are summarized into four folds. (1) Taxonomies of concepts, challenges, practices, techniques, and approaches for testing autonomous systems. (2) A workbench with interconnected tools and a workflow for end-to-end critical scenario identification, which is demonstrated on two industrial autonomous driving systems. (3) Taxonomies of industry practices, initiatives, and challenges for identifying and using critical scenarios for testing autonomous driving systems. (4) A methodology to evaluate the realism of synthetic critical scenarios and a proof of concept of it on two autonomous vehicle collision scenario sets. <br/><br/>Conclusion: We explore the industry practices of and design an effective approach for critical scenario identification as well as a methodology to evaluate the realism of the resulting scenarios. The current practices of critical scenario identification are limited and hindered by several challenges such as cost, tool support, and scenario realism. Therefore, we recommend the industry and academia combine existing approaches, collaborate, and continuously learn to improve their practical use for testing of autonomous driving systems.}},
author = {{Song, Qunying}},
isbn = {{978-91-8104-243-6}},
keywords = {{Autonomous driving systems; software testing; Search-based testing; Simulation; Critical scenario; Critical scenario identification; Industry practices; Challenges; Realism}},
language = {{eng}},
publisher = {{Computer Science, Lund University}},
school = {{Lund University}},
title = {{Critical Scenario Identification for Testing of Realistic Autonomous Driving Systems}},
url = {{https://lup.lub.lu.se/search/files/197587787/PhD_thesis_Qunying_Song.pdf}},
year = {{2024}},
}