BERT based natural language processing for triage of adverse drug reaction reports shows close to human-level performance
Bergman, Erik ; Dürlich, Luise ; Arthurson, Veronica ; Sundström, Anders ; Larsson, Maria ; Bhuiyan, Shamima ; Jakobsson, Andreas LU
and Westman, Gabriel
(2023)
In PLOS Digital Health
2(12).
- Abstract
Post-marketing reports of suspected adverse drug reactions are important for establishing the safety profile of a medicinal product. However, a high influx of reports poses a challenge for regulatory authorities as a delay in identification of previously unknown adverse drug reactions can potentially be harmful to patients. In this study, we use natural language processing (NLP) to predict whether a report is of serious nature based solely on the free-text fields and adverse event terms in the report, potentially allowing reports mislabelled at time of reporting to be detected and prioritized for assessment. We consider four different NLP models at various levels of complexity, bootstrap their train-validation data split to eliminate... (More)
Post-marketing reports of suspected adverse drug reactions are important for establishing the safety profile of a medicinal product. However, a high influx of reports poses a challenge for regulatory authorities as a delay in identification of previously unknown adverse drug reactions can potentially be harmful to patients. In this study, we use natural language processing (NLP) to predict whether a report is of serious nature based solely on the free-text fields and adverse event terms in the report, potentially allowing reports mislabelled at time of reporting to be detected and prioritized for assessment. We consider four different NLP models at various levels of complexity, bootstrap their train-validation data split to eliminate random effects in the performance estimates and conduct prospective testing to avoid the risk of data leakage. Using a Swedish BERT based language model, continued language pre-training and final classification training, we achieve close to human-level performance in this task. Model architectures based on less complex technical foundation such as bag-of-words approaches and LSTM neural networks trained with random initiation of weights appear to perform less well, likely due to the lack of robustness that a base of general language training provides.
(Less)
- author
- Bergman, Erik
; Dürlich, Luise
; Arthurson, Veronica
; Sundström, Anders
; Larsson, Maria
; Bhuiyan, Shamima
; Jakobsson, Andreas
LU
and Westman, Gabriel
- organization
- publishing date
- 2023-12
- type
- Contribution to journal
- publication status
- published
- subject
- in
- PLOS Digital Health
- volume
- 2
- issue
- 12
- article number
- e0000409
- publisher
- Public Library of Science
- external identifiers
-
- pmid:38055685
- scopus:85201972066
- DOI
- 10.1371/journal.pdig.0000409
- language
- English
- LU publication?
- yes
- id
- bf8de97b-e460-40ef-917c-a455ef39941c
- date added to LUP
- 2024-11-01 15:38:26
- date last changed
- 2025-07-13 03:13:17
@article{bf8de97b-e460-40ef-917c-a455ef39941c,
abstract = {{<p>Post-marketing reports of suspected adverse drug reactions are important for establishing the safety profile of a medicinal product. However, a high influx of reports poses a challenge for regulatory authorities as a delay in identification of previously unknown adverse drug reactions can potentially be harmful to patients. In this study, we use natural language processing (NLP) to predict whether a report is of serious nature based solely on the free-text fields and adverse event terms in the report, potentially allowing reports mislabelled at time of reporting to be detected and prioritized for assessment. We consider four different NLP models at various levels of complexity, bootstrap their train-validation data split to eliminate random effects in the performance estimates and conduct prospective testing to avoid the risk of data leakage. Using a Swedish BERT based language model, continued language pre-training and final classification training, we achieve close to human-level performance in this task. Model architectures based on less complex technical foundation such as bag-of-words approaches and LSTM neural networks trained with random initiation of weights appear to perform less well, likely due to the lack of robustness that a base of general language training provides.</p>}},
author = {{Bergman, Erik and Dürlich, Luise and Arthurson, Veronica and Sundström, Anders and Larsson, Maria and Bhuiyan, Shamima and Jakobsson, Andreas and Westman, Gabriel}},
language = {{eng}},
number = {{12}},
publisher = {{Public Library of Science}},
series = {{PLOS Digital Health}},
title = {{BERT based natural language processing for triage of adverse drug reaction reports shows close to human-level performance}},
url = {{http://dx.doi.org/10.1371/journal.pdig.0000409}},
doi = {{10.1371/journal.pdig.0000409}},
volume = {{2}},
year = {{2023}},
}