Discrete-time neural Markov models
Sundell, Jesper LU ; Wahlquist, Ylva LU and Soltesz, Kristian LU
(2026)
In BMC Medical Research Methodology
26(21).
- Abstract
- Background: Markov models are models used to describe movement of individuals between states. In medicine, Markov models are used to describe, for example, disease progression and the effects of interventions on such progression. The models can further be used to identify medical risk groups and aid clinicians in clinical decision making. Although accurate individual predictions are crucial for utility on an individual level, modeling of relationships between covariates and transition probabilities includes assumptions regarding the distribution of the data and the functional form of the relationship limiting accurate prediction of individual risks. The purpose of this work was to develop a flexible framework for discrete-time Markov... (More)
- Background: Markov models are models used to describe movement of individuals between states. In medicine, Markov models are used to describe, for example, disease progression and the effects of interventions on such progression. The models can further be used to identify medical risk groups and aid clinicians in clinical decision making. Although accurate individual predictions are crucial for utility on an individual level, modeling of relationships between covariates and transition probabilities includes assumptions regarding the distribution of the data and the functional form of the relationship limiting accurate prediction of individual risks. The purpose of this work was to develop a flexible framework for discrete-time Markov models enabling the use of neural networks to predict individual transition probabilities.
Methods: The framework was implemented in Julia. Both real-world data and simulated data including two possible competing transitions, censoring and non-linear covariate relationships was used for model evaluation. Three models were evalauted: a model excluding covariates, a model including linear mappings between covariates and transition probabilities, and a model using neural networks for predicting individual transition probabilities based on covariates. Models were trained by minimizing the negative likelihood using mini-batching of data for efficient computations of gradients.
Results: The neural network model outperformed the linear model in predicting individual transition probabilities. Both the negative likelihood and the mean squared error between observed and predicted individual transition probabilities were lower for the neural network model than for the linear model. Individual predicted transition probabilities were more evenly distributed around the line of identity for the neural network model than for the linear model indicating a better predicted functional form of the covariate mapping.
Conclusion: A simple and flexible framework for development of discrete-time Markov models is described. Including neural networks to map the relationship between covariates and transition probabilities enables increased accuracy in predictions of individual risks and provides a tool to inform clinical interventions. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/a1939668-cb52-452a-bc8e-ae70eccdf039
- author
- Sundell, Jesper
LU
; Wahlquist, Ylva
LU
and Soltesz, Kristian
LU
- organization
- publishing date
- 2026
- type
- Contribution to journal
- publication status
- published
- subject
- in
- BMC Medical Research Methodology
- volume
- 26
- issue
- 21
- publisher
- BioMed Central (BMC)
- external identifiers
-
- pmid:41572163
- scopus:105028991993
- ISSN
- 1471-2288
- DOI
- 10.1186/s12874-026-02769-5
- project
- Learning pharmacometric model structures from data
- language
- English
- LU publication?
- yes
- id
- a1939668-cb52-452a-bc8e-ae70eccdf039
- date added to LUP
- 2026-01-14 08:33:24
- date last changed
- 2026-04-11 04:00:49
@article{a1939668-cb52-452a-bc8e-ae70eccdf039,
abstract = {{Background: Markov models are models used to describe movement of individuals between states. In medicine, Markov models are used to describe, for example, disease progression and the effects of interventions on such progression. The models can further be used to identify medical risk groups and aid clinicians in clinical decision making. Although accurate individual predictions are crucial for utility on an individual level, modeling of relationships between covariates and transition probabilities includes assumptions regarding the distribution of the data and the functional form of the relationship limiting accurate prediction of individual risks. The purpose of this work was to develop a flexible framework for discrete-time Markov models enabling the use of neural networks to predict individual transition probabilities.<br/>Methods: The framework was implemented in Julia. Both real-world data and simulated data including two possible competing transitions, censoring and non-linear covariate relationships was used for model evaluation. Three models were evalauted: a model excluding covariates, a model including linear mappings between covariates and transition probabilities, and a model using neural networks for predicting individual transition probabilities based on covariates. Models were trained by minimizing the negative likelihood using mini-batching of data for efficient computations of gradients. <br/>Results: The neural network model outperformed the linear model in predicting individual transition probabilities. Both the negative likelihood and the mean squared error between observed and predicted individual transition probabilities were lower for the neural network model than for the linear model. Individual predicted transition probabilities were more evenly distributed around the line of identity for the neural network model than for the linear model indicating a better predicted functional form of the covariate mapping.<br/>Conclusion: A simple and flexible framework for development of discrete-time Markov models is described. Including neural networks to map the relationship between covariates and transition probabilities enables increased accuracy in predictions of individual risks and provides a tool to inform clinical interventions.}},
author = {{Sundell, Jesper and Wahlquist, Ylva and Soltesz, Kristian}},
issn = {{1471-2288}},
language = {{eng}},
number = {{21}},
publisher = {{BioMed Central (BMC)}},
series = {{BMC Medical Research Methodology}},
title = {{Discrete-time neural Markov models}},
url = {{https://lup.lub.lu.se/search/files/239407998/Discrete_time_neural_Markov_models_accepted_manuscript.pdf}},
doi = {{10.1186/s12874-026-02769-5}},
volume = {{26}},
year = {{2026}},
}