Advanced

Construction of the Berkeley Innovation Index: A Higher-Order Item Response Theory Model Approach

Eng Larsson, Johan and Fred Ojala, Alexander (2016) FMS820 20162
Mathematical Statistics
Abstract
The Berkeley Innovation Index (BII) is a tool developed for assessment of individual innovation capability. The index is based on responses to a survey that constitutes of questions linked to domain abilities, i.e., sub-traits, that are hypothesized to govern an individual's overall innovation ability. The underlying algorithm for the BII produces a score representing the test-takers' proficiency on the domain ability continua as well as a score associated with their general innovation ability. In this thesis, the algorithm for the BII is constructed by applying a Higher-Order Item Response Theory model for hierarchical latent trait estimation. Simultaneous estimation of the vast amount of model parameters is done by employing a Markov... (More)
The Berkeley Innovation Index (BII) is a tool developed for assessment of individual innovation capability. The index is based on responses to a survey that constitutes of questions linked to domain abilities, i.e., sub-traits, that are hypothesized to govern an individual's overall innovation ability. The underlying algorithm for the BII produces a score representing the test-takers' proficiency on the domain ability continua as well as a score associated with their general innovation ability. In this thesis, the algorithm for the BII is constructed by applying a Higher-Order Item Response Theory model for hierarchical latent trait estimation. Simultaneous estimation of the vast amount of model parameters is done by employing a Markov Chain Monte Carlo (MCMC) method that utilizes a multi-level bayesian inference sampling technique. The validity, feasibility, and usefulness of the approach is analyzed throughout the thesis. The statistical relevance of the obtained results is evaluated by examining the Deviance Information Criteria, the Item Response Theory Information Criteria, the posterior predictive values, different convergence criteria for the MCMC chains etc. In order to reduce the amount of questions, and make the index more user-friendly, feature selection techniques are applied to explore the possibility of discarding items that contribute with the least amount of information. An easily implementable and scalable algorithm is presented, and the advantages/disadvantages of the acquired model are discussed. Lastly, recommendations on how to further improve the Berkeley Innovation Index are proposed. (Less)
Please use this url to cite or link to this publication:
author
Eng Larsson, Johan and Fred Ojala, Alexander
supervisor
organization
course
FMS820 20162
year
type
H2 - Master's Degree (Two Years)
subject
keywords
Index Construction, Innovation Capability, Item Response Theory, Higher-Order Latent Trait Estimation, Bayesian Inference, Markov Chain Monte Carlo, Feature Selection
language
English
id
8895254
date added to LUP
2016-11-22 08:01:04
date last changed
2016-12-12 09:36:16
@misc{8895254,
  abstract     = {The Berkeley Innovation Index (BII) is a tool developed for assessment of individual innovation capability. The index is based on responses to a survey that constitutes of questions linked to domain abilities, i.e., sub-traits, that are hypothesized to govern an individual's overall innovation ability. The underlying algorithm for the BII produces a score representing the test-takers' proficiency on the domain ability continua as well as a score associated with their general innovation ability. In this thesis, the algorithm for the BII is constructed by applying a Higher-Order Item Response Theory model for hierarchical latent trait estimation. Simultaneous estimation of the vast amount of model parameters is done by employing a Markov Chain Monte Carlo (MCMC) method that utilizes a multi-level bayesian inference sampling technique. The validity, feasibility, and usefulness of the approach is analyzed throughout the thesis. The statistical relevance of the obtained results is evaluated by examining the Deviance Information Criteria, the Item Response Theory Information Criteria, the posterior predictive values, different convergence criteria for the MCMC chains etc. In order to reduce the amount of questions, and make the index more user-friendly, feature selection techniques are applied to explore the possibility of discarding items that contribute with the least amount of information. An easily implementable and scalable algorithm is presented, and the advantages/disadvantages of the acquired model are discussed. Lastly, recommendations on how to further improve the Berkeley Innovation Index are proposed.},
  author       = {Eng Larsson, Johan and Fred Ojala, Alexander},
  keyword      = {Index Construction,Innovation Capability,Item Response Theory,Higher-Order Latent Trait Estimation,Bayesian Inference,Markov Chain Monte Carlo,Feature Selection},
  language     = {eng},
  note         = {Student Paper},
  title        = {Construction of the Berkeley Innovation Index: A Higher-Order Item Response Theory Model Approach},
  year         = {2016},
}