Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Training deep learning based dynamic MR image reconstruction using open-source natural videos

Jaubert, Olivier ; Pascale, Michele ; Montalt-Tordera, Javier ; Akesson, Julius LU ; Virsinskaite, Ruta ; Knight, Daniel ; Arridge, Simon ; Steeden, Jennifer and Muthurangu, Vivek (2024) In Scientific Reports 14(1).
Abstract

To develop and assess a deep learning (DL) pipeline to learn dynamic MR image reconstruction from publicly available natural videos (Inter4K). Learning was performed for a range of DL architectures (VarNet, 3D UNet, FastDVDNet) and corresponding sampling patterns (Cartesian, radial, spiral) either from true multi-coil cardiac MR data (N = 692) or from synthetic MR data simulated from Inter4K natural videos (N = 588). Real-time undersampled dynamic MR images were reconstructed using DL networks trained with cardiac data and natural videos, and compressed sensing (CS). Differences were assessed in simulations (N = 104 datasets) in terms of MSE, PSNR, and SSIM and prospectively for cardiac cine (short axis, four chambers, N = 20) and... (More)

To develop and assess a deep learning (DL) pipeline to learn dynamic MR image reconstruction from publicly available natural videos (Inter4K). Learning was performed for a range of DL architectures (VarNet, 3D UNet, FastDVDNet) and corresponding sampling patterns (Cartesian, radial, spiral) either from true multi-coil cardiac MR data (N = 692) or from synthetic MR data simulated from Inter4K natural videos (N = 588). Real-time undersampled dynamic MR images were reconstructed using DL networks trained with cardiac data and natural videos, and compressed sensing (CS). Differences were assessed in simulations (N = 104 datasets) in terms of MSE, PSNR, and SSIM and prospectively for cardiac cine (short axis, four chambers, N = 20) and speech cine (N = 10) data in terms of subjective image quality ranking, SNR and Edge sharpness. Friedman Chi Square tests with post-hoc Nemenyi analysis were performed to assess statistical significance. In simulated data, DL networks trained with cardiac data outperformed DL networks trained with natural videos, both of which outperformed CS (p < 0.05). However, in prospective experiments DL reconstructions using both training datasets were ranked similarly (and higher than CS) and presented no statistical differences in SNR and Edge Sharpness for most conditions.The developed pipeline enabled learning dynamic MR reconstruction from natural videos preserving DL reconstruction advantages such as high quality fast and ultra-fast reconstructions while overcoming some limitations (data scarcity or sharing). The natural video dataset, code and pre-trained networks are made readily available on github.

(Less)
Please use this url to cite or link to this publication:
author
; ; ; ; ; ; ; and
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
Deep learning, Dynamic MRI, Image reconstruction, Machine learning, Real-time
in
Scientific Reports
volume
14
issue
1
article number
11774
publisher
Nature Publishing Group
external identifiers
  • scopus:85194127703
  • pmid:38783018
ISSN
2045-2322
DOI
10.1038/s41598-024-62294-7
language
English
LU publication?
yes
id
36a60e4c-82f9-44d3-820b-8e601d3aa6be
date added to LUP
2024-06-05 14:25:31
date last changed
2024-06-19 15:16:11
@article{36a60e4c-82f9-44d3-820b-8e601d3aa6be,
  abstract     = {{<p>To develop and assess a deep learning (DL) pipeline to learn dynamic MR image reconstruction from publicly available natural videos (Inter4K). Learning was performed for a range of DL architectures (VarNet, 3D UNet, FastDVDNet) and corresponding sampling patterns (Cartesian, radial, spiral) either from true multi-coil cardiac MR data (N = 692) or from synthetic MR data simulated from Inter4K natural videos (N = 588). Real-time undersampled dynamic MR images were reconstructed using DL networks trained with cardiac data and natural videos, and compressed sensing (CS). Differences were assessed in simulations (N = 104 datasets) in terms of MSE, PSNR, and SSIM and prospectively for cardiac cine (short axis, four chambers, N = 20) and speech cine (N = 10) data in terms of subjective image quality ranking, SNR and Edge sharpness. Friedman Chi Square tests with post-hoc Nemenyi analysis were performed to assess statistical significance. In simulated data, DL networks trained with cardiac data outperformed DL networks trained with natural videos, both of which outperformed CS (p &lt; 0.05). However, in prospective experiments DL reconstructions using both training datasets were ranked similarly (and higher than CS) and presented no statistical differences in SNR and Edge Sharpness for most conditions.The developed pipeline enabled learning dynamic MR reconstruction from natural videos preserving DL reconstruction advantages such as high quality fast and ultra-fast reconstructions while overcoming some limitations (data scarcity or sharing). The natural video dataset, code and pre-trained networks are made readily available on github.</p>}},
  author       = {{Jaubert, Olivier and Pascale, Michele and Montalt-Tordera, Javier and Akesson, Julius and Virsinskaite, Ruta and Knight, Daniel and Arridge, Simon and Steeden, Jennifer and Muthurangu, Vivek}},
  issn         = {{2045-2322}},
  keywords     = {{Deep learning; Dynamic MRI; Image reconstruction; Machine learning; Real-time}},
  language     = {{eng}},
  number       = {{1}},
  publisher    = {{Nature Publishing Group}},
  series       = {{Scientific Reports}},
  title        = {{Training deep learning based dynamic MR image reconstruction using open-source natural videos}},
  url          = {{http://dx.doi.org/10.1038/s41598-024-62294-7}},
  doi          = {{10.1038/s41598-024-62294-7}},
  volume       = {{14}},
  year         = {{2024}},
}