Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

AI Act High-Risk Requirements Readiness : Industrial Perspectives and Case Company Insights

Wagner, Matthias LU orcid ; Gupta, Rushali LU ; Borg, Markus LU ; Engström, Emelie LU orcid and Lysek, Michal (2024) 25th International Conference on Product-Focused Software Process Improvement, PROFES 2024 In Lecture Notes in Computer Science (LNCS) 15453. p.67-83
Abstract
The AI Act’s (AIA) requirements for high-risk AI systems affect many aspects of modern software systems. Knowing which AIA-related technical challenges are relevant to different companies is essential to focus compliance-oriented research on the aspects that matter. We therefore conducted an interview study in collaboration with a case company that specializes in network video solutions within the security and surveillance industry. External experts enrich the study for a broader industry perspective. The goal was to analyze the case company’s readiness for the AIA’s high-risk requirements, based on methods and techniques already established prior to the legislation. Our results yielded a positive sentiment towards the regulation and the... (More)
The AI Act’s (AIA) requirements for high-risk AI systems affect many aspects of modern software systems. Knowing which AIA-related technical challenges are relevant to different companies is essential to focus compliance-oriented research on the aspects that matter. We therefore conducted an interview study in collaboration with a case company that specializes in network video solutions within the security and surveillance industry. External experts enrich the study for a broader industry perspective. The goal was to analyze the case company’s readiness for the AIA’s high-risk requirements, based on methods and techniques already established prior to the legislation. Our results yielded a positive sentiment towards the regulation and the planning security that it brings, although a high workload was expected. We identified a solid foundation with well-established practices to build upon for the requirements on cybersecurity, human oversight, record-keeping, and technical documentation. However, we also report several open challenges, mainly connected to the requirement on data quality and governance, followed by accuracy, robustness, and cybersecurity. The AIA specifically demands a post-market monitoring system (Art 72) and the right to an explanation of individual decision-making (Art 86). These two obligations were identified as especially challenging by the respondents. The result of this study is expected to steer future compliance-oriented work toward pressing challenges. (Less)
Please use this url to cite or link to this publication:
author
; ; ; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
AI Act, Interview Study, Regulatory Requirements
host publication
Product-Focused Software Process Improvement : 25th International Conference, PROFES 2024, Tartu, Estonia, December 02–04, 2024, Proceedings - 25th International Conference, PROFES 2024, Tartu, Estonia, December 02–04, 2024, Proceedings
series title
Lecture Notes in Computer Science (LNCS)
editor
Pfahl, Dietmar ; Huerta, Javier Gonzalez ; Klünder, Jil and Anwar, Hina
volume
15453
pages
16 pages
publisher
Springer
conference name
25th International Conference on Product-Focused Software Process Improvement, PROFES 2024
conference location
Tartu, Estonia
conference dates
2024-12-02 - 2024-12-04
external identifiers
  • scopus:85211217697
ISSN
0302-9743
1611-3349
ISBN
978-3-031-78392-0
978-3-031-78391-3
DOI
10.1007/978-3-031-78392-0_5
language
English
LU publication?
yes
id
c7101acc-3fa6-41b6-b623-c9062e0008ec
date added to LUP
2024-11-11 17:23:28
date last changed
2025-04-28 09:39:50
@inproceedings{c7101acc-3fa6-41b6-b623-c9062e0008ec,
  abstract     = {{The AI Act’s (AIA) requirements for high-risk AI systems affect many aspects of modern software systems. Knowing which AIA-related technical challenges are relevant to different companies is essential to focus compliance-oriented research on the aspects that matter. We therefore conducted an interview study in collaboration with a case company that specializes in network video solutions within the security and surveillance industry. External experts enrich the study for a broader industry perspective. The goal was to analyze the case company’s readiness for the AIA’s high-risk requirements, based on methods and techniques already established prior to the legislation. Our results yielded a positive sentiment towards the regulation and the planning security that it brings, although a high workload was expected. We identified a solid foundation with well-established practices to build upon for the requirements on cybersecurity, human oversight, record-keeping, and technical documentation. However, we also report several open challenges, mainly connected to the requirement on data quality and governance, followed by accuracy, robustness, and cybersecurity. The AIA specifically demands a post-market monitoring system (Art 72) and the right to an explanation of individual decision-making (Art 86). These two obligations were identified as especially challenging by the respondents. The result of this study is expected to steer future compliance-oriented work toward pressing challenges.}},
  author       = {{Wagner, Matthias and Gupta, Rushali and Borg, Markus and Engström, Emelie and Lysek, Michal}},
  booktitle    = {{Product-Focused Software Process Improvement : 25th International Conference, PROFES 2024, Tartu, Estonia, December 02–04, 2024, Proceedings}},
  editor       = {{Pfahl, Dietmar and Huerta, Javier Gonzalez and Klünder, Jil and Anwar, Hina}},
  isbn         = {{978-3-031-78392-0}},
  issn         = {{0302-9743}},
  keywords     = {{AI Act; Interview Study; Regulatory Requirements}},
  language     = {{eng}},
  month        = {{11}},
  pages        = {{67--83}},
  publisher    = {{Springer}},
  series       = {{Lecture Notes in Computer Science (LNCS)}},
  title        = {{AI Act High-Risk Requirements Readiness : Industrial Perspectives and Case Company Insights}},
  url          = {{http://dx.doi.org/10.1007/978-3-031-78392-0_5}},
  doi          = {{10.1007/978-3-031-78392-0_5}},
  volume       = {{15453}},
  year         = {{2024}},
}