AI Act high-risk requirements readiness : industrial perspectives and case company insights
Wagner, Matthias LU
; Gupta, Rushali
LU
; Borg, Markus
LU
; Engström, Emelie
LU
and Lysek, Michal
(2024)
25th International Conference on Product-Focused Software Process Improvement, PROFES 2024
In Lecture Notes in Computer Science (LNCS)
15453.
p.67-83
- Abstract
- The AI Act’s (AIA) requirements for high-risk AI systems affect many aspects of modern software systems. Knowing which AIA-related technical challenges are relevant to different companies is essential to focus compliance-oriented research on the aspects that matter. We therefore conducted an interview study in collaboration with a case company that specializes in network video solutions within the security and surveillance industry. External experts enrich the study for a broader industry perspective. The goal was to analyze the case company’s readiness for the AIA’s high-risk requirements, based on methods and techniques already established prior to the legislation. Our results yielded a positive sentiment towards the regulation and the... (More)
- The AI Act’s (AIA) requirements for high-risk AI systems affect many aspects of modern software systems. Knowing which AIA-related technical challenges are relevant to different companies is essential to focus compliance-oriented research on the aspects that matter. We therefore conducted an interview study in collaboration with a case company that specializes in network video solutions within the security and surveillance industry. External experts enrich the study for a broader industry perspective. The goal was to analyze the case company’s readiness for the AIA’s high-risk requirements, based on methods and techniques already established prior to the legislation. Our results yielded a positive sentiment towards the regulation and the planning security that it brings, although a high workload was expected. We identified a solid foundation with well-established practices to build upon for the requirements on cybersecurity, human oversight, record-keeping, and technical documentation. However, we also report several open challenges, mainly connected to the requirement on data quality and governance, followed by accuracy, robustness, and cybersecurity. The AIA specifically demands a post-market monitoring system (Art 72) and the right to an explanation of individual decision-making (Art 86). These two obligations were identified as especially challenging by the respondents. The result of this study is expected to steer future compliance-oriented work toward pressing challenges. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/c7101acc-3fa6-41b6-b623-c9062e0008ec
- author
- Wagner, Matthias
LU
; Gupta, Rushali
LU
; Borg, Markus
LU
; Engström, Emelie
LU
and Lysek, Michal
- organization
- publishing date
- 2024-11-27
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- keywords
- AI Act, Interview study, Regulatory requirements
- host publication
- Product-Focused Software Process Improvement : 25th International Conference, PROFES 2024, Tartu, Estonia, December 02–04, 2024, Proceedings - 25th International Conference, PROFES 2024, Tartu, Estonia, December 02–04, 2024, Proceedings
- series title
- Lecture Notes in Computer Science (LNCS)
- editor
- Pfahl, Dietmar ; Huerta, Javier Gonzalez ; Klünder, Jil and Anwar, Hina
- volume
- 15453
- pages
- 16 pages
- publisher
- Springer
- conference name
- 25th International Conference on Product-Focused Software Process Improvement, PROFES 2024
- conference location
- Tartu, Estonia
- conference dates
- 2024-12-02 - 2024-12-04
- external identifiers
-
- scopus:85211217697
- ISSN
- 0302-9743
- 1611-3349
- ISBN
- 978-3-031-78392-0
- 978-3-031-78391-3
- DOI
- 10.1007/978-3-031-78392-0_5
- language
- English
- LU publication?
- yes
- id
- c7101acc-3fa6-41b6-b623-c9062e0008ec
- date added to LUP
- 2024-11-11 17:23:28
- date last changed
- 2025-05-26 12:36:06
@inproceedings{c7101acc-3fa6-41b6-b623-c9062e0008ec,
abstract = {{The AI Act’s (AIA) requirements for high-risk AI systems affect many aspects of modern software systems. Knowing which AIA-related technical challenges are relevant to different companies is essential to focus compliance-oriented research on the aspects that matter. We therefore conducted an interview study in collaboration with a case company that specializes in network video solutions within the security and surveillance industry. External experts enrich the study for a broader industry perspective. The goal was to analyze the case company’s readiness for the AIA’s high-risk requirements, based on methods and techniques already established prior to the legislation. Our results yielded a positive sentiment towards the regulation and the planning security that it brings, although a high workload was expected. We identified a solid foundation with well-established practices to build upon for the requirements on cybersecurity, human oversight, record-keeping, and technical documentation. However, we also report several open challenges, mainly connected to the requirement on data quality and governance, followed by accuracy, robustness, and cybersecurity. The AIA specifically demands a post-market monitoring system (Art 72) and the right to an explanation of individual decision-making (Art 86). These two obligations were identified as especially challenging by the respondents. The result of this study is expected to steer future compliance-oriented work toward pressing challenges.}},
author = {{Wagner, Matthias and Gupta, Rushali and Borg, Markus and Engström, Emelie and Lysek, Michal}},
booktitle = {{Product-Focused Software Process Improvement : 25th International Conference, PROFES 2024, Tartu, Estonia, December 02–04, 2024, Proceedings}},
editor = {{Pfahl, Dietmar and Huerta, Javier Gonzalez and Klünder, Jil and Anwar, Hina}},
isbn = {{978-3-031-78392-0}},
issn = {{0302-9743}},
keywords = {{AI Act; Interview study; Regulatory requirements}},
language = {{eng}},
month = {{11}},
pages = {{67--83}},
publisher = {{Springer}},
series = {{Lecture Notes in Computer Science (LNCS)}},
title = {{AI Act high-risk requirements readiness : industrial perspectives and case company insights}},
url = {{http://dx.doi.org/10.1007/978-3-031-78392-0_5}},
doi = {{10.1007/978-3-031-78392-0_5}},
volume = {{15453}},
year = {{2024}},
}