Cross-validation of optimized composites for preclinical Alzheimer's disease
(2017) In Alzheimer's and Dementia: Translational Research and Clinical Interventions 3(1). p.123-129- Abstract
Introduction We discuss optimization and validation of composite end points for presymptomatic Alzheimer's disease clinical trials. Optimized composites offer hope of substantial gains in statistical power or reduction in sample size. But there is tradeoff between optimization and face validity such that optimization should only be considered if there is a convincing rationale. As with statistically derived regions of interest in neuroimaging, validation on independent data sets is essential. Methods Using four data sets, we consider the optimized weighting of four components of a cognitive composite which includes measures of (1) global cognition, (2) semantic memory, (3) episodic memory, and (4) executive function. Weights are... (More)
Introduction We discuss optimization and validation of composite end points for presymptomatic Alzheimer's disease clinical trials. Optimized composites offer hope of substantial gains in statistical power or reduction in sample size. But there is tradeoff between optimization and face validity such that optimization should only be considered if there is a convincing rationale. As with statistically derived regions of interest in neuroimaging, validation on independent data sets is essential. Methods Using four data sets, we consider the optimized weighting of four components of a cognitive composite which includes measures of (1) global cognition, (2) semantic memory, (3) episodic memory, and (4) executive function. Weights are optimized to either discriminate amyloid positivity or maximize power to detect a treatment effect in an amyloid-positive population. We apply repeated 5 × 3-fold cross-validation to quantify the out-of-sample performance of optimized composite end points. Results We found the optimized weights varied greatly across the folds of the cross-validation with either optimization method. Both optimization methods tend to down-weight the measures of global cognition and executive function. However, when these optimized composites were applied to the validation sets, they did not provide consistent improvements in power. In fact, overall, the optimized composites performed worse than those without optimization. Discussion We find that component weight optimization does not yield valid improvements in sensitivity of this composite to detect treatment effects.
(Less)
- author
- Donohue, Michael C. ; Sun, Chung Kai ; Raman, Rema ; Insel, Philip S. LU and Aisen, Paul S.
- organization
- publishing date
- 2017-01-01
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- Cognitive composites, End-point validation, Preclinical Alzheimer's disease
- in
- Alzheimer's and Dementia: Translational Research and Clinical Interventions
- volume
- 3
- issue
- 1
- pages
- 7 pages
- publisher
- John Wiley & Sons Inc.
- external identifiers
-
- pmid:28758145
- scopus:85013449855
- DOI
- 10.1016/j.trci.2016.12.001
- language
- English
- LU publication?
- yes
- id
- 72f4e57a-511d-4326-989a-b4a47af114fb
- date added to LUP
- 2017-03-24 14:10:35
- date last changed
- 2024-05-26 12:41:37
@article{72f4e57a-511d-4326-989a-b4a47af114fb,
abstract = {{<p>Introduction We discuss optimization and validation of composite end points for presymptomatic Alzheimer's disease clinical trials. Optimized composites offer hope of substantial gains in statistical power or reduction in sample size. But there is tradeoff between optimization and face validity such that optimization should only be considered if there is a convincing rationale. As with statistically derived regions of interest in neuroimaging, validation on independent data sets is essential. Methods Using four data sets, we consider the optimized weighting of four components of a cognitive composite which includes measures of (1) global cognition, (2) semantic memory, (3) episodic memory, and (4) executive function. Weights are optimized to either discriminate amyloid positivity or maximize power to detect a treatment effect in an amyloid-positive population. We apply repeated 5 × 3-fold cross-validation to quantify the out-of-sample performance of optimized composite end points. Results We found the optimized weights varied greatly across the folds of the cross-validation with either optimization method. Both optimization methods tend to down-weight the measures of global cognition and executive function. However, when these optimized composites were applied to the validation sets, they did not provide consistent improvements in power. In fact, overall, the optimized composites performed worse than those without optimization. Discussion We find that component weight optimization does not yield valid improvements in sensitivity of this composite to detect treatment effects.</p>}},
author = {{Donohue, Michael C. and Sun, Chung Kai and Raman, Rema and Insel, Philip S. and Aisen, Paul S.}},
keywords = {{Cognitive composites; End-point validation; Preclinical Alzheimer's disease}},
language = {{eng}},
month = {{01}},
number = {{1}},
pages = {{123--129}},
publisher = {{John Wiley & Sons Inc.}},
series = {{Alzheimer's and Dementia: Translational Research and Clinical Interventions}},
title = {{Cross-validation of optimized composites for preclinical Alzheimer's disease}},
url = {{http://dx.doi.org/10.1016/j.trci.2016.12.001}},
doi = {{10.1016/j.trci.2016.12.001}},
volume = {{3}},
year = {{2017}},
}