Can Machine Learning Help Pharmacovigilance See Warning Signs Earlier? : Disproportionality analysis and machine learning prioritization of pancreatitis reports associated with GLP-1 receptor agonists in FAERS, with regulatory considerations

Abstract

Glucagon-like peptide-1 receptor agonists (GLP-1 RAs) are increasingly used to treat type 2 diabetes mellitus and obesity. As their use expands, continued post-marketing safety monitoring is important, especially for uncommon but serious adverse events. Despite conflicting evidence regarding causality, pancreatitis, in particular, has remained a clinical and regulatory concern for GLP-1 drugs. This thesis evaluated pancreatitis reports associated with GLP-1 receptor agonists in the U.S. Food and Drug Administration Adverse Event Reporting System (FAERS) from 2015 to 2024.

The study used a two-phase pharmacovigilance framework. First, classical and Bayesian disproportionality methods were applied to assess whether pancreatitis was reported more often than expected for the GLP-1 RA class and for individual agents. Second, supervised machine learning was evaluated as a complementary tool for prioritizing GLP-1 RA-exposed individual case safety reports for review. Logistic regression and histogram-based gradient boosting were compared using discrimination, top-k prioritization, temporal validation, calibration, and recalibration analyses.

The disproportionality analyses identified positive pancreatitis signals for all seven GLP-1 receptor agonists examined, with concordant findings across proportional reporting ratio, reporting odds ratio, information component, and empirical Bayes geometric mean analyses. Temporal analyses showed that the contribution of individual drugs changed over time, with older agents contributing more strongly in earlier years and newer agents becoming more prominent later. In the primary machine-learning analysis, gradient boosting performed better than logistic regression for ranking and top-k report prioritization. However, under stricter temporal validation, machine-learning performance was substantially weaker, indicating limited robustness when models were tested on later reports.

Overall, the findings support a complementary pharmacovigilance framework in which disproportionality methods remain central for signal detection, while machine learning may help prioritize reports for expert review. However, the results also show that machine-learning outputs require cautious interpretation, especially with respect to calibration, temporal stability, and generalizability. These models should therefore be viewed as decision-support tools rather than re-placements for pharmacovigilance expertise.

Popular Abstract

Medicines undergo rigorous testing before they are made available to the public, however, rare side effects may only become visible after they are widely used. But how do we know if a medicine is causing problems once it reaches the market? One important clue comes from large safety databases, where doctors, pharmaceutical companies and sometimes patients report suspected side effects. These reports do not prove that a drug caused harm, but they can reveal patterns that deserve closer attention.

This project focused on GLP-1 receptor agonists, a group of medicines used to treat type 2 diabetes and obesity. Because these drugs are being used by more and more people, long-term safety monitoring has become increasingly important. Here was examined whether reports of pancreatitis, a potentially serious inflammation of the pancreas, appeared unusually often together with these medicines in the US Food and Drug Administration Adverse Event Reporting System, known as FAERS.

The results showed a clear warning signal. Across all seven GLP-1 drugs included in the study, pancreatitis was reported more often than expected. This does not mean the drugs were proven to cause pancreatitis, but it does mean the pattern was strong enough to justify continued attention from researchers and regulators. It was also found that the pattern changed over time. Older GLP-1 drugs contributed more to the reported adverse events in earlier years, while newer drugs became more prominent later in terms of reported adverse events. In other words, the safety landscape did not stand still.

A second question was then asked: can artificial intelligence help sort through large numbers of safety reports? Instead of trying to replace traditional safety methods in the field of pharmacovigilance, it was tested whether machine learning could help prioritize which reports looked most relevant for expert review. In the strongest analysis, one model performed better than a simpler baseline at ranking reports likely to contain pancreatitis. That sounds promising, but there was an important catch. When the models were tested in a stricter, more forward-looking way, their performance became much weaker.

That may be the most important lesson from the project. In drug safety work, AI is probably most useful as an assistant, not as an automatic decision-maker. Traditional pharmacovigilance methods are still essential for detecting warning signals, while machine learning may help experts decide which reports to read first when time and resources are limited. The safest approach is therefore not 'AI instead of humans', but 'AI together with human expertise'.

This project shows both the promise and the limits of AI in pharmacovigilance. Machine learning may support faster review of important safety reports, but only when it is used carefully, transparently and with strong human oversight. For medicines that millions of people depend on, better safety monitoring matters. Even a small improvement in how warnings are detected and prioritized can make a difference.

• author: Balidemaj, Festina
• supervisor: Stephen Burleigh
• organization: Pharmaceutical Technology (master)
• course: KLGM06 20261
• year: 2026
• type: H2 - Master's Degree (Two Years)
• subject: Technology and Engineering; Biology and Life Sciences
• keywords: pharmacovigilance, FAERS, GLP-1 receptor agonists, pancreatitis, disproportionality analysis, machine learning, temporal validation, report prioritization, pharmaceutical formulation
• language: English
• id: 9231311
• date added to LUP: 2026-06-16 14:41:59
• date last changed: 2026-07-03 15:00:48