Utility effectiveness, composite indicators and ‘benefit of the doubt’ models : How important are preferences when evaluating the performance of innovation systems?
(2026) In Socio-Economic Planning Sciences 106.- Abstract
This study introduces a methodology that embeds ‘Benefit of the Doubt’ evaluations of performance of innovation systems, based on objective measures of technical effectiveness—that compare the values of a set of indicators across countries—into the literature of composite indicators, which rank countries relying on utility functions and whose aggregating structure and weights are subjectively chosen by the decision makers. Considering the general case of the constant elasticity of substitution utility function, whose power mean formulation nests popular specifications such as the weighted (linear) arithmetic mean, weighted geometric mean (Cobb-Douglas), and fixed-proportions (Leontief), we define the concept of utility effectiveness of... (More)
This study introduces a methodology that embeds ‘Benefit of the Doubt’ evaluations of performance of innovation systems, based on objective measures of technical effectiveness—that compare the values of a set of indicators across countries—into the literature of composite indicators, which rank countries relying on utility functions and whose aggregating structure and weights are subjectively chosen by the decision makers. Considering the general case of the constant elasticity of substitution utility function, whose power mean formulation nests popular specifications such as the weighted (linear) arithmetic mean, weighted geometric mean (Cobb-Douglas), and fixed-proportions (Leontief), we define the concept of utility effectiveness of an alternative as the ratio of its observed utility to the maximum utility observed across countries. We show that the utility loss resulting from an ineffective performance can be consistently decomposed into two mutually exclusive factors: first, grounded in the objective criterion of Pareto-dominance, the mentioned technical effectiveness is represented by a technical (radial) measure depending only on quantities, which gauges the distance to the utility frontier; second, a measure of allocative effectiveness, determining the additional utility loss resulting from an unbalanced composition of indicators given the preferences of the decision maker. Using data from the European Innovation Scoreboard, our results show high levels of utility ineffectiveness in Southern and Eastern European countries. On average, these countries' performance is half of the maximum utility reached by the best performing innovation systems. Objective technical ineffectiveness generally dominates over subjective allocative ineffectiveness (except for the fixed proportion specification), showing that the practical impact of choosing one utility specification over another, although sizable, is limited. Consequently, main performance deficits are often not due to substitutability or compensatory issues, but rather to real shortfalls in indicator levels, while preference-driven differences tend to be second-order and comparatively robust.
(Less)
- author
- Zofío, José Luis ; Aparicio, Juan ; Barbero, Javier and Zabala-Iturriagagoitia, Jon Mikel LU
- organization
- publishing date
- 2026-08
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- Benefit-of-the-doubt evaluation, Composite indicators, Innovation performance, Innovation systems, Utility effectiveness
- in
- Socio-Economic Planning Sciences
- volume
- 106
- article number
- 102528
- publisher
- Elsevier
- external identifiers
-
- scopus:105043044082
- ISSN
- 0038-0121
- DOI
- 10.1016/j.seps.2026.102528
- language
- English
- LU publication?
- yes
- id
- 1fda1fb3-4a13-4fa6-82d7-97ab31ea79a8
- date added to LUP
- 2026-10-01 15:24:10
- date last changed
- 2026-10-01 15:24:56
@article{1fda1fb3-4a13-4fa6-82d7-97ab31ea79a8,
abstract = {{<p>This study introduces a methodology that embeds ‘Benefit of the Doubt’ evaluations of performance of innovation systems, based on objective measures of technical effectiveness—that compare the values of a set of indicators across countries—into the literature of composite indicators, which rank countries relying on utility functions and whose aggregating structure and weights are subjectively chosen by the decision makers. Considering the general case of the constant elasticity of substitution utility function, whose power mean formulation nests popular specifications such as the weighted (linear) arithmetic mean, weighted geometric mean (Cobb-Douglas), and fixed-proportions (Leontief), we define the concept of utility effectiveness of an alternative as the ratio of its observed utility to the maximum utility observed across countries. We show that the utility loss resulting from an ineffective performance can be consistently decomposed into two mutually exclusive factors: first, grounded in the objective criterion of Pareto-dominance, the mentioned technical effectiveness is represented by a technical (radial) measure depending only on quantities, which gauges the distance to the utility frontier; second, a measure of allocative effectiveness, determining the additional utility loss resulting from an unbalanced composition of indicators given the preferences of the decision maker. Using data from the European Innovation Scoreboard, our results show high levels of utility ineffectiveness in Southern and Eastern European countries. On average, these countries' performance is half of the maximum utility reached by the best performing innovation systems. Objective technical ineffectiveness generally dominates over subjective allocative ineffectiveness (except for the fixed proportion specification), showing that the practical impact of choosing one utility specification over another, although sizable, is limited. Consequently, main performance deficits are often not due to substitutability or compensatory issues, but rather to real shortfalls in indicator levels, while preference-driven differences tend to be second-order and comparatively robust.</p>}},
author = {{Zofío, José Luis and Aparicio, Juan and Barbero, Javier and Zabala-Iturriagagoitia, Jon Mikel}},
issn = {{0038-0121}},
keywords = {{Benefit-of-the-doubt evaluation; Composite indicators; Innovation performance; Innovation systems; Utility effectiveness}},
language = {{eng}},
publisher = {{Elsevier}},
series = {{Socio-Economic Planning Sciences}},
title = {{Utility effectiveness, composite indicators and ‘benefit of the doubt’ models : How important are preferences when evaluating the performance of innovation systems?}},
url = {{http://dx.doi.org/10.1016/j.seps.2026.102528}},
doi = {{10.1016/j.seps.2026.102528}},
volume = {{106}},
year = {{2026}},
}