Advanced

The multi-layered and multilevel use of bibliometric measures in Swedish universities: Isomorphism, translation and strategic choice

Hammarfelt, Björn and Åström, Fredrik LU (2015) Research Organizations Under Scrutiny: the 20th International Conference on Science and Technology Indicators (STI)
Abstract
In recent years, systems for resource allocation in academic research based on performance indicators have been introduced in many European countries (Hicks, 2012). Today, we are at a point where the use of bibliometrics based indicators for research evaluation and resource allocation more or less permeates academic systems on all levels and across the world. The implementation—and the effects of the use—of bibliometric indicators on the micro- and macro-levels have been studied, but the exploration of the use of bibliometric indicators for resource allocation in academic research on the meso-level—that is within HEIs, between faculties and departments—has so far been limited, albeit with some exceptions (Aagaard 2015; Hammarfelt & de... (More)
In recent years, systems for resource allocation in academic research based on performance indicators have been introduced in many European countries (Hicks, 2012). Today, we are at a point where the use of bibliometrics based indicators for research evaluation and resource allocation more or less permeates academic systems on all levels and across the world. The implementation—and the effects of the use—of bibliometric indicators on the micro- and macro-levels have been studied, but the exploration of the use of bibliometric indicators for resource allocation in academic research on the meso-level—that is within HEIs, between faculties and departments—has so far been limited, albeit with some exceptions (Aagaard 2015; Hammarfelt & de Rijcke 2015).

This study provides a comprehensive overview and analysis of the local use of bibliometric indicators at Swedish universities through a survey directed at 26 Swedish institutions of higher education and our theoretical focus is inspired by new institutionalism theories (DiMaggio & Powell, 1983).

Findings

Bibliometric measurement is currently applied in 24 of the 26 HEIs in our study. There is large variation in terms of what kind of bibliometric indicator being used for resource allocation. Generally, we can identify three main types of indicators: those counting number of publications, those based on citation frequency, and those making calculations based on a combination of citation and publication counts (Table 1).

As with the choice of publication or citation based indicators, we find a wide variation in terms of on which levels bibliometric indicators are used for distributing resources at different HEIs. Seven out of the 26 HEIs in our sample are use bibliometric indicators to allocate resources directly to individuals. The models used at Blekinge Institute of Technology and Luleå University of Technology are perhaps the most straightforward ones as they directly allocate resources—in the form of research time or travel money—based on articles published in journals indexed in Web of Science or rated at prestige level two in the Norwegian system.

Discussion

Performance based allocation based on bibliometric indicators has become the norm in Swedish academia. From a viewpoint of ‘new institutionalism’ we could describe the current focus on bibliometric measurement at Swedish HEIs as a result of isomorphism. Still, even if many allocation models resemble each other we find that a range of different indicators is used on various levels.

Hence, it is evident that national systems are translated, negotiated and possible also contested on the local level. We also see clear evidence for strategic choices being made when indicators and systems are chosen. Finally we argue that in-depth and systematic knowledge about the actual use of bibliometrics across all levels of academia is pre-requisite for studying the effect of measurement. Further studies might, help us to understand how indicators on all levels, from university rankings to evaluation of individual researchers, reinforce, interact and contradict each other in the forming of a ‘metric’ culture in academia. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to conference
publication status
published
subject
keywords
Swedish Universities, Bibliometrics, Research evaluation systems
pages
10 pages
conference name
Research Organizations Under Scrutiny: the 20th International Conference on Science and Technology Indicators (STI)
language
English
LU publication?
yes
id
42013b83-651f-4619-a782-d1d0e086c078 (old id 7864175)
alternative location
http://www.sti2015.usi.ch/sites/www.sti2015.usi.ch/files/media/hammarfelt_and_astrom_sti_2015.pdf
date added to LUP
2015-09-10 12:50:28
date last changed
2016-10-18 08:45:36
@misc{42013b83-651f-4619-a782-d1d0e086c078,
  abstract     = {In recent years, systems for resource allocation in academic research based on performance indicators have been introduced in many European countries (Hicks, 2012). Today, we are at a point where the use of bibliometrics based indicators for research evaluation and resource allocation more or less permeates academic systems on all levels and across the world. The implementation—and the effects of the use—of bibliometric indicators on the micro- and macro-levels have been studied, but the exploration of the use of bibliometric indicators for resource allocation in academic research on the meso-level—that is within HEIs, between faculties and departments—has so far been limited, albeit with some exceptions (Aagaard 2015; Hammarfelt &amp; de Rijcke 2015).<br/><br>
This study provides a comprehensive overview and analysis of the local use of bibliometric indicators at Swedish universities through a survey directed at 26 Swedish institutions of higher education and our theoretical focus is inspired by new institutionalism theories (DiMaggio &amp; Powell, 1983).<br/><br>
Findings<br/><br>
Bibliometric measurement is currently applied in 24 of the 26 HEIs in our study. There is large variation in terms of what kind of bibliometric indicator being used for resource allocation. Generally, we can identify three main types of indicators: those counting number of publications, those based on citation frequency, and those making calculations based on a combination of citation and publication counts (Table 1).<br/><br>
As with the choice of publication or citation based indicators, we find a wide variation in terms of on which levels bibliometric indicators are used for distributing resources at different HEIs. Seven out of the 26 HEIs in our sample are use bibliometric indicators to allocate resources directly to individuals. The models used at Blekinge Institute of Technology and Luleå University of Technology are perhaps the most straightforward ones as they directly allocate resources—in the form of research time or travel money—based on articles published in journals indexed in Web of Science or rated at prestige level two in the Norwegian system.<br/><br>
Discussion<br/><br>
Performance based allocation based on bibliometric indicators has become the norm in Swedish academia. From a viewpoint of ‘new institutionalism’ we could describe the current focus on bibliometric measurement at Swedish HEIs as a result of isomorphism. Still, even if many allocation models resemble each other we find that a range of different indicators is used on various levels.<br/><br>
Hence, it is evident that national systems are translated, negotiated and possible also contested on the local level. We also see clear evidence for strategic choices being made when indicators and systems are chosen. Finally we argue that in-depth and systematic knowledge about the actual use of bibliometrics across all levels of academia is pre-requisite for studying the effect of measurement. Further studies might, help us to understand how indicators on all levels, from university rankings to evaluation of individual researchers, reinforce, interact and contradict each other in the forming of a ‘metric’ culture in academia.},
  author       = {Hammarfelt, Björn and Åström, Fredrik},
  keyword      = {Swedish Universities,Bibliometrics,Research evaluation systems},
  language     = {eng},
  pages        = {10},
  title        = {The multi-layered and multilevel use of bibliometric measures in Swedish universities: Isomorphism, translation and strategic choice},
  year         = {2015},
}