Advanced

Problems with reliability of speech variables for use in quality registries for cleft lip and palate—experiences from the swedish cleft lip and palate registry

Malmborn, Jan Olof; Becker, Magnus LU and Klintö, Kristina LU (2018) In Cleft Palate-Craniofacial Journal 55(8). p.1051-1059
Abstract

Objective: To test the reliability of the speech data in the Swedish quality registry for cleft lip and palate. Design: Retrospective study. Setting: Primary care university hospital. Participants: Ninety-four children born with cleft palate with or without cleft lip between 2005 and 2009 who had been assessed and registered in the quality registry at the age of 5 years. Main Outcome Measures: Data in the registry on percent oral consonants correct, percent oral errors, percent nonoral errors, perceived velopharyngeal function (PVPF), and intelligibility were compared with results based on reassessments by 3 independent raters from audio recordings. Agreement was calculated by the intraclass correlation coefficient (ICC), quadratic... (More)

Objective: To test the reliability of the speech data in the Swedish quality registry for cleft lip and palate. Design: Retrospective study. Setting: Primary care university hospital. Participants: Ninety-four children born with cleft palate with or without cleft lip between 2005 and 2009 who had been assessed and registered in the quality registry at the age of 5 years. Main Outcome Measures: Data in the registry on percent oral consonants correct, percent oral errors, percent nonoral errors, perceived velopharyngeal function (PVPF), and intelligibility were compared with results based on reassessments by 3 independent raters from audio recordings. Agreement was calculated by the intraclass correlation coefficient (ICC), quadratic weighted kappa, and percentage agreement. Results: Absolute agreement calculated by average measures ICC for percent oral consonants correct, percent oral errors, and percent nonoral errors was above >0.90. Single measures ICC for percent oral consonants correct was 0.82, for percent oral errors 0.69, and for percent nonoral errors 0.83. The kappa coefficient for PVPF was 0.5 to 0.59 and for intelligibility 0.65 to 0.77. Exact percentage agreement for PVPF was 33% and for intelligibility 47.8%. Conclusions: The data on oral consonants correct and nonoral errors in the quality registry seem to be reliable. The data on oral errors, PVPF, and intelligibility should be interpreted with caution. If differences among treatment centers are detected, one should go back and examine the collected raw data before drawing any definitive conclusions about treatment outcome.

(Less)
Please use this url to cite or link to this publication:
author
publishing date
type
Contribution to journal
publication status
published
subject
keywords
Cleft lip and palate, Quality registry, Reliability, Speech evaluation, Validity
in
Cleft Palate-Craniofacial Journal
volume
55
issue
8
pages
9 pages
publisher
Allen Press
external identifiers
  • scopus:85055579845
ISSN
1055-6656
DOI
10.1177/1055665618765777
language
English
LU publication?
no
id
e63b6d1d-32db-49d0-949e-121936962c09
date added to LUP
2018-11-20 08:52:53
date last changed
2019-01-06 14:16:15
@article{e63b6d1d-32db-49d0-949e-121936962c09,
  abstract     = {<p>Objective: To test the reliability of the speech data in the Swedish quality registry for cleft lip and palate. Design: Retrospective study. Setting: Primary care university hospital. Participants: Ninety-four children born with cleft palate with or without cleft lip between 2005 and 2009 who had been assessed and registered in the quality registry at the age of 5 years. Main Outcome Measures: Data in the registry on percent oral consonants correct, percent oral errors, percent nonoral errors, perceived velopharyngeal function (PVPF), and intelligibility were compared with results based on reassessments by 3 independent raters from audio recordings. Agreement was calculated by the intraclass correlation coefficient (ICC), quadratic weighted kappa, and percentage agreement. Results: Absolute agreement calculated by average measures ICC for percent oral consonants correct, percent oral errors, and percent nonoral errors was above &gt;0.90. Single measures ICC for percent oral consonants correct was 0.82, for percent oral errors 0.69, and for percent nonoral errors 0.83. The kappa coefficient for PVPF was 0.5 to 0.59 and for intelligibility 0.65 to 0.77. Exact percentage agreement for PVPF was 33% and for intelligibility 47.8%. Conclusions: The data on oral consonants correct and nonoral errors in the quality registry seem to be reliable. The data on oral errors, PVPF, and intelligibility should be interpreted with caution. If differences among treatment centers are detected, one should go back and examine the collected raw data before drawing any definitive conclusions about treatment outcome.</p>},
  author       = {Malmborn, Jan Olof and Becker, Magnus and Klintö, Kristina},
  issn         = {1055-6656},
  keyword      = {Cleft lip and palate,Quality registry,Reliability,Speech evaluation,Validity},
  language     = {eng},
  number       = {8},
  pages        = {1051--1059},
  publisher    = {Allen Press},
  series       = {Cleft Palate-Craniofacial Journal},
  title        = {Problems with reliability of speech variables for use in quality registries for cleft lip and palate—experiences from the swedish cleft lip and palate registry},
  url          = {http://dx.doi.org/10.1177/1055665618765777},
  volume       = {55},
  year         = {2018},
}