Advanced

Intra- and Inter-Rater Agreement Describing Myometrial Lesions Using Morphologic Uterus Sonographic Assessment : A Pilot Study

Rasmussen, Christina Kjaergaard ; Van den Bosch, Thierry ; Exacoustos, Caterina ; Manegold-Brauer, Gwendolin ; Benacerraf, Beryl R. ; Froyman, Wouter ; Landolfo, Chiara ; Condorelli, Margherita ; Egekvist, Anne G. and Josefsson, Hampus , et al. (2019) In Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine 38(10). p.2673-2683
Abstract

OBJECTIVES: To evaluate the intra- and inter-rater agreement for myometrial lesions using Morphologic Uterus Sonographic Assessment terminology. METHODS: Thirteen raters with high (n = 6) or medium experience (n = 7) assessed 30 3-dimensional ultrasound clips with (n = 20) and without (n = 10) benign myometrial lesions. Myometrial lesions were reported as poorly or well defined and then systematically evaluated for the presence of individual features. The clips were blindly assessed twice (at a 2-month interval). Intra- and inter-rater agreements were calculated with κ statistics. RESULTS: The reporting of poorly defined lesions reached moderate intra-rater agreement (κ = 0.49 [high experience] and 0.47 [medium experience]) and poor... (More)

OBJECTIVES: To evaluate the intra- and inter-rater agreement for myometrial lesions using Morphologic Uterus Sonographic Assessment terminology. METHODS: Thirteen raters with high (n = 6) or medium experience (n = 7) assessed 30 3-dimensional ultrasound clips with (n = 20) and without (n = 10) benign myometrial lesions. Myometrial lesions were reported as poorly or well defined and then systematically evaluated for the presence of individual features. The clips were blindly assessed twice (at a 2-month interval). Intra- and inter-rater agreements were calculated with κ statistics. RESULTS: The reporting of poorly defined lesions reached moderate intra-rater agreement (κ = 0.49 [high experience] and 0.47 [medium experience]) and poor inter-rater agreement (κ = 0.39 [high experience] and 0.25 [medium experience]). The reporting of well-defined lesions reached good to very good intra-rater agreement (κ = 0.73 [high experience] and 0.82 [medium experience]) and good inter-rater agreement (κ = 0.75 [high experience] and 0.63 [medium experience]). Most individual features associated with ill-defined lesions reached moderate intra- and inter-rater agreement among highly experienced raters (κ = 0.41-0.60). The least reproducible features were myometrial cysts, hyperechoic islands, subendometrial lines and buds, and translesional flow (κ = 0.11-0.34). Most individual features associated with well-defined lesions reached moderate to good intra- and inter-rater agreement among all observers (κ = 0.41-0.80). The least reproducible features were a serosal contour, asymmetry, a hyperechoic rim, and fan-shaped shadows (κ = 0.00-0.35). CONCLUSIONS: The reporting of well-defined lesions showed excellent agreement, whereas the agreement for poorly defined lesions was low, even among highly experienced raters. The agreement on identifying individual features varied, especially for features associated with ill-defined lesions. Guidelines on minimum requirements for features associated with ill-defined lesions to be interpreted as poorly defined lesions may improve agreement.

(Less)
Please use this url to cite or link to this publication:
author
, et al. (More)
(Less)
publishing date
type
Contribution to journal
publication status
published
subject
keywords
adenomyosis, fibromas, gynecology, observer variability, ultrasound
in
Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine
volume
38
issue
10
pages
11 pages
publisher
American Institute of Ultrasound in Medicine
external identifiers
  • scopus:85072314822
  • pmid:30801764
ISSN
1550-9613
DOI
10.1002/jum.14971
language
English
LU publication?
no
id
6da2a38f-d89c-4ddd-ba80-37a81289bbd1
date added to LUP
2019-09-30 12:53:00
date last changed
2019-12-10 08:15:32
@article{6da2a38f-d89c-4ddd-ba80-37a81289bbd1,
  abstract     = {<p>OBJECTIVES: To evaluate the intra- and inter-rater agreement for myometrial lesions using Morphologic Uterus Sonographic Assessment terminology. METHODS: Thirteen raters with high (n = 6) or medium experience (n = 7) assessed 30 3-dimensional ultrasound clips with (n = 20) and without (n = 10) benign myometrial lesions. Myometrial lesions were reported as poorly or well defined and then systematically evaluated for the presence of individual features. The clips were blindly assessed twice (at a 2-month interval). Intra- and inter-rater agreements were calculated with κ statistics. RESULTS: The reporting of poorly defined lesions reached moderate intra-rater agreement (κ = 0.49 [high experience] and 0.47 [medium experience]) and poor inter-rater agreement (κ = 0.39 [high experience] and 0.25 [medium experience]). The reporting of well-defined lesions reached good to very good intra-rater agreement (κ = 0.73 [high experience] and 0.82 [medium experience]) and good inter-rater agreement (κ = 0.75 [high experience] and 0.63 [medium experience]). Most individual features associated with ill-defined lesions reached moderate intra- and inter-rater agreement among highly experienced raters (κ = 0.41-0.60). The least reproducible features were myometrial cysts, hyperechoic islands, subendometrial lines and buds, and translesional flow (κ = 0.11-0.34). Most individual features associated with well-defined lesions reached moderate to good intra- and inter-rater agreement among all observers (κ = 0.41-0.80). The least reproducible features were a serosal contour, asymmetry, a hyperechoic rim, and fan-shaped shadows (κ = 0.00-0.35). CONCLUSIONS: The reporting of well-defined lesions showed excellent agreement, whereas the agreement for poorly defined lesions was low, even among highly experienced raters. The agreement on identifying individual features varied, especially for features associated with ill-defined lesions. Guidelines on minimum requirements for features associated with ill-defined lesions to be interpreted as poorly defined lesions may improve agreement.</p>},
  author       = {Rasmussen, Christina Kjaergaard and Van den Bosch, Thierry and Exacoustos, Caterina and Manegold-Brauer, Gwendolin and Benacerraf, Beryl R. and Froyman, Wouter and Landolfo, Chiara and Condorelli, Margherita and Egekvist, Anne G. and Josefsson, Hampus and Leone, Francesco Paolo Giuseppe and Jokubkiene, Ligita and Zannoni, Letizia and Epstein, Elisabeth and Installé, Arnaud and Dueholm, Margit},
  issn         = {1550-9613},
  language     = {eng},
  number       = {10},
  pages        = {2673--2683},
  publisher    = {American Institute of Ultrasound in Medicine},
  series       = {Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine},
  title        = {Intra- and Inter-Rater Agreement Describing Myometrial Lesions Using Morphologic Uterus Sonographic Assessment : A Pilot Study},
  url          = {http://dx.doi.org/10.1002/jum.14971},
  doi          = {10.1002/jum.14971},
  volume       = {38},
  year         = {2019},
}