Robust analysis of uncertainty in scientific assessments
(2021) Abstract
 Uncertainty refers to any limitation in knowledge. Identifying and characterizing uncertainty in conclusions is important to ensure transparency and avoid over or under confidence in scientific assessments. Quantitative expressions of uncertainty are less ambiguous compared to uncertainty expressed qualitatively, or not at all. Subjective probability is an example of a quantitative expression of epistemic uncertainty, which combined with Bayesian inference makes it possible to integrate evidence and characterizes uncertainty in quantities of interest. This thesis contributes to the understanding and implementation of robust Bayesian analysis as a way to integrate expert judgment and data into assessments and quantify uncertainty by bounded... (More)
 Uncertainty refers to any limitation in knowledge. Identifying and characterizing uncertainty in conclusions is important to ensure transparency and avoid over or under confidence in scientific assessments. Quantitative expressions of uncertainty are less ambiguous compared to uncertainty expressed qualitatively, or not at all. Subjective probability is an example of a quantitative expression of epistemic uncertainty, which combined with Bayesian inference makes it possible to integrate evidence and characterizes uncertainty in quantities of interest. This thesis contributes to the understanding and implementation of robust Bayesian analysis as a way to integrate expert judgment and data into assessments and quantify uncertainty by bounded probability. The robust Bayesian framework is based on sets of probability for epistemic uncertainty, where precise probability is seen as a special case. This thesis covers applications relevant for scientific assessments, including evidence synthesis and quantitative risk assessment.
Paper I proposes to combine two sampling methods: iterative importance sampling and Markov chain Monte Carlo (MCMC) sampling, for quantifying uncertainty by bounded probability when Bayesian updating requires MCMC sampling. This opens up for robust Bayesian analysis to be applied to complex statistical models. To achieve this, an effective sample size of importance sampling that accounts for correlated MCMC samples is proposed. For illustration, the proposed method is applied to estimate the overall effect with bounded probability in a published metaanalysis within the Collaboration for Environmental Evidence on the effect of biomanipulation on freshwater lakes.
Paper II demonstrates robust Bayesian analysis as a way to quantify uncertainty in a quantity of interest by bounded probability, and explicitly distinguishes between epistemic and aleatory uncertainty in the assessment and learn parameters by integrating evidence into the model. Robust Bayesian analysis is described as a generalization of Bayesian analysis, including Bayesian analysis through precise probability as a special case. Both analyses are applied to an intake assessment.
Paper III describes a way to consider uncertainty arising from ignorance or ambiguity about bias terms in a quantitative bias analysis by characterizing bias with imprecision. This is done by specifying bias with a set of bias terms and use robust Bayesian analysis to estimate the overall effect in the metaanalysis. The approach provides a structured framework to transform qualitative judgments concerning risk of biases into quantitative expressions of uncertainty in quantitative bias analysis.
Paper IV compares the effect of different diversified farming practices on biodiversity and crop yields. This is done by applying a Bayesian network metaanalysis to a new public global database from a systematic protocol on diversified farming. A portfolio analysis calibrated by the network metaanalyses showed that uncertainty about the mean performance is large compared to the variability in performance across different farms.
(Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/b24fd3c2ec9c422ebec265ab62adcb89
 author
 Raices Cruz, Ivette ^{LU}
 supervisor

 Ullrika Sahlin ^{LU}
 Henrik Smith ^{LU}
 Johan LindstrÃ¶m ^{LU}
 NilsEric Sahlin ^{LU}
 Matthias C.M. Troffaes
 opponent

 Professor Borsuk, Mark, Pratt School of Engineering, Duke University, Durham, North Carolina
 organization
 publishing date
 20211122
 type
 Thesis
 publication status
 published
 subject
 keywords
 expert knowledge, robust Bayesian analysis, scientific assessments, subjective probability, uncertainty analysis
 pages
 177 pages
 publisher
 Lund University
 defense location
 Blue Hall, Ecology Building, SÃ¶lvegatan 37, Lund. Join via zoom: https://luse.zoom.us/meeting/register/u5Uucu6vqj4qE9JHvL4goL7C5cohJFY8fdJA (registration is required)
 defense date
 20211217 13:00:00
 ISBN
 9789180391016
 9789180391023
 language
 English
 LU publication?
 yes
 id
 b24fd3c2ec9c422ebec265ab62adcb89
 date added to LUP
 20211122 12:09:52
 date last changed
 20220627 14:34:41
@phdthesis{b24fd3c2ec9c422ebec265ab62adcb89, abstract = {{Uncertainty refers to any limitation in knowledge. Identifying and characterizing uncertainty in conclusions is important to ensure transparency and avoid over or under confidence in scientific assessments. Quantitative expressions of uncertainty are less ambiguous compared to uncertainty expressed qualitatively, or not at all. Subjective probability is an example of a quantitative expression of epistemic uncertainty, which combined with Bayesian inference makes it possible to integrate evidence and characterizes uncertainty in quantities of interest. This thesis contributes to the understanding and implementation of robust Bayesian analysis as a way to integrate expert judgment and data into assessments and quantify uncertainty by bounded probability. The robust Bayesian framework is based on sets of probability for epistemic uncertainty, where precise probability is seen as a special case. This thesis covers applications relevant for scientific assessments, including evidence synthesis and quantitative risk assessment.<br/><br/>Paper I proposes to combine two sampling methods: iterative importance sampling and Markov chain Monte Carlo (MCMC) sampling, for quantifying uncertainty by bounded probability when Bayesian updating requires MCMC sampling. This opens up for robust Bayesian analysis to be applied to complex statistical models. To achieve this, an effective sample size of importance sampling that accounts for correlated MCMC samples is proposed. For illustration, the proposed method is applied to estimate the overall effect with bounded probability in a published metaanalysis within the Collaboration for Environmental Evidence on the effect of biomanipulation on freshwater lakes.<br/><br/>Paper II demonstrates robust Bayesian analysis as a way to quantify uncertainty in a quantity of interest by bounded probability, and explicitly distinguishes between epistemic and aleatory uncertainty in the assessment and learn parameters by integrating evidence into the model. Robust Bayesian analysis is described as a generalization of Bayesian analysis, including Bayesian analysis through precise probability as a special case. Both analyses are applied to an intake assessment.<br/><br/>Paper III describes a way to consider uncertainty arising from ignorance or ambiguity about bias terms in a quantitative bias analysis by characterizing bias with imprecision. This is done by specifying bias with a set of bias terms and use robust Bayesian analysis to estimate the overall effect in the metaanalysis. The approach provides a structured framework to transform qualitative judgments concerning risk of biases into quantitative expressions of uncertainty in quantitative bias analysis.<br/><br/>Paper IV compares the effect of different diversified farming practices on biodiversity and crop yields. This is done by applying a Bayesian network metaanalysis to a new public global database from a systematic protocol on diversified farming. A portfolio analysis calibrated by the network metaanalyses showed that uncertainty about the mean performance is large compared to the variability in performance across different farms.<br/>}}, author = {{Raices Cruz, Ivette}}, isbn = {{9789180391016}}, keywords = {{expert knowledge; robust Bayesian analysis; scientific assessments; subjective probability; uncertainty analysis}}, language = {{eng}}, month = {{11}}, publisher = {{Lund University}}, school = {{Lund University}}, title = {{Robust analysis of uncertainty in scientific assessments}}, url = {{https://lup.lub.lu.se/search/files/110000826/PhD_thesis_WEBB_kappa.pdf}}, year = {{2021}}, }