Industry Practices for Challenging Autonomous Driving Systems with Critical Scenarios
Song, Qunying LU
; Engström, Emelie
LU
and Runeson, Per
LU
(2024)
In ACM Transactions on Software Engineering and Methodology
33(4).
- Abstract
Testing autonomous driving systems for safety and reliability is essential, yet complex. A primary challenge is identifying relevant test scenarios, especially the critical ones that may expose hazards or harm to autonomous vehicles and other road users. Although numerous approaches and tools for critical scenario identification are proposed, the industry practices for selection, implementation, and evaluation of approaches, are not well understood. Therefore, we aim at exploring practical aspects of how autonomous driving systems are tested, particularly the identification and use of critical scenarios. We interviewed 13 practitioners from 7 companies in autonomous driving in Sweden. We used thematic modeling to analyse and synthesize... (More)
Testing autonomous driving systems for safety and reliability is essential, yet complex. A primary challenge is identifying relevant test scenarios, especially the critical ones that may expose hazards or harm to autonomous vehicles and other road users. Although numerous approaches and tools for critical scenario identification are proposed, the industry practices for selection, implementation, and evaluation of approaches, are not well understood. Therefore, we aim at exploring practical aspects of how autonomous driving systems are tested, particularly the identification and use of critical scenarios. We interviewed 13 practitioners from 7 companies in autonomous driving in Sweden. We used thematic modeling to analyse and synthesize the interview data. As a result, we present 9 themes of practices and 4 themes of challenges related to critical scenarios. Our analysis indicates there is little joint effort in the industry, despite every approach has its own limitations, and tools and platforms are lacking. To that end, we recommend the industry and academia combine different approaches, collaborate among different stakeholders, and continuously learn the field. The contributions of our study are exploration and synthesis of industry practices and related challenges for critical scenario identification and testing, and potential increase of industry relevance for future studies.
(Less)
- author
- Song, Qunying
LU
; Engström, Emelie
LU
and Runeson, Per
LU
- organization
- publishing date
- 2024-04-18
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- autonomous driving systems, challenges, Critical scenario identification, industry practices, interview, testing
- in
- ACM Transactions on Software Engineering and Methodology
- volume
- 33
- issue
- 4
- article number
- 99
- publisher
- Association for Computing Machinery (ACM)
- external identifiers
-
- scopus:85191562794
- ISSN
- 1049-331X
- DOI
- 10.1145/3640334
- language
- English
- LU publication?
- yes
- additional info
- Publisher Copyright: © 2024 Copyright held by the owner/author(s).
- id
- 8b0da6c6-5892-450c-bc63-70bf8a02bedd
- date added to LUP
- 2024-05-13 08:05:23
- date last changed
- 2024-05-13 18:07:40
@article{8b0da6c6-5892-450c-bc63-70bf8a02bedd,
abstract = {{<p>Testing autonomous driving systems for safety and reliability is essential, yet complex. A primary challenge is identifying relevant test scenarios, especially the critical ones that may expose hazards or harm to autonomous vehicles and other road users. Although numerous approaches and tools for critical scenario identification are proposed, the industry practices for selection, implementation, and evaluation of approaches, are not well understood. Therefore, we aim at exploring practical aspects of how autonomous driving systems are tested, particularly the identification and use of critical scenarios. We interviewed 13 practitioners from 7 companies in autonomous driving in Sweden. We used thematic modeling to analyse and synthesize the interview data. As a result, we present 9 themes of practices and 4 themes of challenges related to critical scenarios. Our analysis indicates there is little joint effort in the industry, despite every approach has its own limitations, and tools and platforms are lacking. To that end, we recommend the industry and academia combine different approaches, collaborate among different stakeholders, and continuously learn the field. The contributions of our study are exploration and synthesis of industry practices and related challenges for critical scenario identification and testing, and potential increase of industry relevance for future studies.</p>}},
author = {{Song, Qunying and Engström, Emelie and Runeson, Per}},
issn = {{1049-331X}},
keywords = {{autonomous driving systems; challenges; Critical scenario identification; industry practices; interview; testing}},
language = {{eng}},
month = {{04}},
number = {{4}},
publisher = {{Association for Computing Machinery (ACM)}},
series = {{ACM Transactions on Software Engineering and Methodology}},
title = {{Industry Practices for Challenging Autonomous Driving Systems with Critical Scenarios}},
url = {{http://dx.doi.org/10.1145/3640334}},
doi = {{10.1145/3640334}},
volume = {{33}},
year = {{2024}},
}