Advanced

Review and new evidence on composite innovation indicators for evaluating national performance

Grupp, Hariolf and Schubert, Torben LU (2010) In Research Policy 39(1). p.67-78
Abstract

The purpose of this contribution is to present a survey of the recent developments in constructing composite science and technology (S&T) indicators on a national level as well as new evidence of the variability of such S&T indicators which opens the gateway to "country-tuning". It has become standard practice to combine several indicators for science, technology, and innovation to form composite numbers. Especially in the light of this variability, two questions arise. Firstly, are the results (especially rankings) stable with respect to weights? Secondly, is there hope to define "economically" reasonable weights? In order to provide answers to these questions, we use data from the European Innovation Scoreboard 2005 (EIS 2005)... (More)

The purpose of this contribution is to present a survey of the recent developments in constructing composite science and technology (S&T) indicators on a national level as well as new evidence of the variability of such S&T indicators which opens the gateway to "country-tuning". It has become standard practice to combine several indicators for science, technology, and innovation to form composite numbers. Especially in the light of this variability, two questions arise. Firstly, are the results (especially rankings) stable with respect to weights? Secondly, is there hope to define "economically" reasonable weights? In order to provide answers to these questions, we use data from the European Innovation Scoreboard 2005 (EIS 2005) to exemplify our reasoning. Concerning the first question, we give genuine evidence on the existence of immense variability, possibly invalidating the results. Further, we also show that even existing and well-accepted methods, like equal weighting, Benefit of the Doubt weighting (BoD) and principal component analysis weighting (PCA) may lead to drastically differing results. Concerning the second question we will demonstrate that by each composite indicator weighting a set of shadow prices is implied expressing one indicator in terms of another. Whether the weights are sensible should be evaluated on the basis of these shadow prices. It turns out that those implied by EIS 2005 contain strange peculiarities. After that we plead for more care in constructing composite indicators. Especially weights should be chosen on the basis of shadow prices, rather than, say, by equal weighting or other automatic methods. Lastly, we discuss the merit of composite indicators and argue that they have a valuable communication and competition function, but they should be accompanied by multidimensional representations, which provide the basis for the construction of policy measures. © 2009 Elsevier B.V. All rights reserved.

(Less)
Please use this url to cite or link to this publication:
author
publishing date
type
Contribution to journal
publication status
published
subject
keywords
Composite indicators, National innovation systems, Science and technology indicators, Scoreboards
in
Research Policy
volume
39
issue
1
pages
12 pages
publisher
Elsevier
external identifiers
  • Scopus:74149093952
ISSN
0048-7333
DOI
10.1016/j.respol.2009.10.002
language
English
LU publication?
no
id
d28f824e-8107-4537-96b7-a77159510267
date added to LUP
2016-05-18 13:33:04
date last changed
2017-01-01 08:26:15
@article{d28f824e-8107-4537-96b7-a77159510267,
  abstract     = {<p>The purpose of this contribution is to present a survey of the recent developments in constructing composite science and technology (S&amp;T) indicators on a national level as well as new evidence of the variability of such S&amp;T indicators which opens the gateway to "country-tuning". It has become standard practice to combine several indicators for science, technology, and innovation to form composite numbers. Especially in the light of this variability, two questions arise. Firstly, are the results (especially rankings) stable with respect to weights? Secondly, is there hope to define "economically" reasonable weights? In order to provide answers to these questions, we use data from the European Innovation Scoreboard 2005 (EIS 2005) to exemplify our reasoning. Concerning the first question, we give genuine evidence on the existence of immense variability, possibly invalidating the results. Further, we also show that even existing and well-accepted methods, like equal weighting, Benefit of the Doubt weighting (BoD) and principal component analysis weighting (PCA) may lead to drastically differing results. Concerning the second question we will demonstrate that by each composite indicator weighting a set of shadow prices is implied expressing one indicator in terms of another. Whether the weights are sensible should be evaluated on the basis of these shadow prices. It turns out that those implied by EIS 2005 contain strange peculiarities. After that we plead for more care in constructing composite indicators. Especially weights should be chosen on the basis of shadow prices, rather than, say, by equal weighting or other automatic methods. Lastly, we discuss the merit of composite indicators and argue that they have a valuable communication and competition function, but they should be accompanied by multidimensional representations, which provide the basis for the construction of policy measures. © 2009 Elsevier B.V. All rights reserved.</p>},
  author       = {Grupp, Hariolf and Schubert, Torben},
  issn         = {0048-7333},
  keyword      = {Composite indicators,National innovation systems,Science and technology indicators,Scoreboards},
  language     = {eng},
  number       = {1},
  pages        = {67--78},
  publisher    = {Elsevier},
  series       = {Research Policy},
  title        = {Review and new evidence on composite innovation indicators for evaluating national performance},
  url          = {http://dx.doi.org/10.1016/j.respol.2009.10.002},
  volume       = {39},
  year         = {2010},
}