Considerations on Performance Evaluation of Atrial Fibrillation Detectors
(2021) In IEEE Transactions on Biomedical Engineering 68(11). p.3250-3260- Abstract
Objective: A large number of atrial fibrillation (AF) detectors have been published in recent years, signifying that the comparison of detector performance plays a central role, though not always consistent. The aim of this study is to shed needed light on aspects crucial to the evaluation of detection performance. Methods: Three types of AF detector, using either information on rhythm, rhythm and morphology, or segments of ECG samples, are implemented and studied on both real and simulated ECG signals. The properties of different performance measures are investigated, for example, in relation to dataset imbalance. Results: The results show that performance can differ considerably depending on the way detector output is compared to... (More)
Objective: A large number of atrial fibrillation (AF) detectors have been published in recent years, signifying that the comparison of detector performance plays a central role, though not always consistent. The aim of this study is to shed needed light on aspects crucial to the evaluation of detection performance. Methods: Three types of AF detector, using either information on rhythm, rhythm and morphology, or segments of ECG samples, are implemented and studied on both real and simulated ECG signals. The properties of different performance measures are investigated, for example, in relation to dataset imbalance. Results: The results show that performance can differ considerably depending on the way detector output is compared to database annotations, i.e., beat-to-beat, segment-to-segment, or episode-to-episode comparison. Moreover, depending on the type of detector, the results substantiate that physiological and technical factors, e.g., changes in ECG morphology, rate of atrial premature beats, and noise level, can have a considerable influence on performance. Conclusion: The present study demonstrates overall strengths and weaknesses of different types of detector, highlights challenges in AF detection, and proposes five recommendations on how to handle data and characterize performance.
(Less)
- author
- Butkuviene, Monika ; Petrenas, Andrius ; Solosenko, Andrius ; Martin-Yebra, Alba ; Marozas, Vaidotas and Sornmo, Leif LU
- organization
- publishing date
- 2021
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- Annotations, Atrial fibrillation, Databases, deep learning, detection, Detectors, Electrocardiography, expert-crafted detection, Monitoring, performance evaluation, performance measures, Rhythm
- in
- IEEE Transactions on Biomedical Engineering
- volume
- 68
- issue
- 11
- pages
- 3250 - 3260
- publisher
- IEEE - Institute of Electrical and Electronics Engineers Inc.
- external identifiers
-
- pmid:33750686
- scopus:85103252943
- ISSN
- 0018-9294
- DOI
- 10.1109/TBME.2021.3067698
- language
- English
- LU publication?
- yes
- id
- 57e7942a-14d7-4c5c-943f-4307fbab3430
- date added to LUP
- 2021-04-08 14:34:05
- date last changed
- 2025-01-26 08:50:14
@article{57e7942a-14d7-4c5c-943f-4307fbab3430,
abstract = {{<p>Objective: A large number of atrial fibrillation (AF) detectors have been published in recent years, signifying that the comparison of detector performance plays a central role, though not always consistent. The aim of this study is to shed needed light on aspects crucial to the evaluation of detection performance. Methods: Three types of AF detector, using either information on rhythm, rhythm and morphology, or segments of ECG samples, are implemented and studied on both real and simulated ECG signals. The properties of different performance measures are investigated, for example, in relation to dataset imbalance. Results: The results show that performance can differ considerably depending on the way detector output is compared to database annotations, i.e., beat-to-beat, segment-to-segment, or episode-to-episode comparison. Moreover, depending on the type of detector, the results substantiate that physiological and technical factors, e.g., changes in ECG morphology, rate of atrial premature beats, and noise level, can have a considerable influence on performance. Conclusion: The present study demonstrates overall strengths and weaknesses of different types of detector, highlights challenges in AF detection, and proposes five recommendations on how to handle data and characterize performance.</p>}},
author = {{Butkuviene, Monika and Petrenas, Andrius and Solosenko, Andrius and Martin-Yebra, Alba and Marozas, Vaidotas and Sornmo, Leif}},
issn = {{0018-9294}},
keywords = {{Annotations; Atrial fibrillation; Databases; deep learning; detection; Detectors; Electrocardiography; expert-crafted detection; Monitoring; performance evaluation; performance measures; Rhythm}},
language = {{eng}},
number = {{11}},
pages = {{3250--3260}},
publisher = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
series = {{IEEE Transactions on Biomedical Engineering}},
title = {{Considerations on Performance Evaluation of Atrial Fibrillation Detectors}},
url = {{http://dx.doi.org/10.1109/TBME.2021.3067698}},
doi = {{10.1109/TBME.2021.3067698}},
volume = {{68}},
year = {{2021}},
}