A critical cluster analysis of 44 indicators of author-level performance

Lorna Elizabeth Wildgaard

doi:10.1016/j.joi.2016.09.003

A critical cluster analysis of 44 indicators of author-level performance

Lorna Elizabeth Wildgaard

Institut for Kommunikation - Forskning

9 Citationer (Scopus)

Abstract

This paper explores a 7-stage cluster methodology as a process to identify appropriate indicators for evaluation of individual researchers at a disciplinary and seniority level. Publication and citation data for 741 researchers from 4 disciplines was collected in Web of Science. Forty-four indicators of individual researcher performance were computed using the data. The clustering solution was supported by continued reference to the researcher’s curriculum vitae, an effect analysis and a risk analysis. Disciplinary appropriate indicators were identified and used to divide the researchers into four groups; low, middle, high and extremely high performers. Seniority-specific indicators were not identified. The practical importance of the recommended disciplinary appropriate indicators is concerning. Our study revealed several critical concerns that should be investigated in the application of statistics in research evaluation.
The strength of the 7-stage cluster methodology is that it makes clear that in the evaluation of individual researchers, statistics cannot stand alone. The methodology is reliant on contextual information to verify the bibliometric values and cluster solution. It is important to do studies that investigate the usefulness of statistical evaluation methodologies to help us as a community learn more about the appropriateness of particular bibliometric indicators in the analysis of different researcher profiles.

Originalsprog	Engelsk
Tidsskrift	Journal of Informetrics
Vol/bind	10
Udgave nummer	4
Sider (fra-til)	1055-1078
Antal sider	24
ISSN	1751-1577
DOI	https://doi.org/10.1016/j.joi.2016.09.003
Status	Udgivet - 1 nov. 2016

Adgang til dokumentet

10.1016/j.joi.2016.09.003

Andre filer og links

preprint_highlights

Citationsformater

@article{182bae09f08b408c80f19c7bb13a09e3,

title = "A critical cluster analysis of 44 indicators of author-level performance",

abstract = "This paper explores a 7-stage cluster methodology as a process to identify appropriate indicators for evaluation of individual researchers at a disciplinary and seniority level. Publication and citation data for 741 researchers from 4 disciplines was collected in Web of Science. Forty-four indicators of individual researcher performance were computed using the data. The clustering solution was supported by continued reference to the researcher{\textquoteright}s curriculum vitae, an effect analysis and a risk analysis. Disciplinary appropriate indicators were identified and used to divide the researchers into four groups; low, middle, high and extremely high performers. Seniority-specific indicators were not identified. The practical importance of the recommended disciplinary appropriate indicators is concerning. Our study revealed several critical concerns that should be investigated in the application of statistics in research evaluation.The strength of the 7-stage cluster methodology is that it makes clear that in the evaluation of individual researchers, statistics cannot stand alone. The methodology is reliant on contextual information to verify the bibliometric values and cluster solution. It is important to do studies that investigate the usefulness of statistical evaluation methodologies to help us as a community learn more about the appropriateness of particular bibliometric indicators in the analysis of different researcher profiles.",

author = "Wildgaard, {Lorna Elizabeth}",

year = "2016",

month = nov,

day = "1",

doi = "10.1016/j.joi.2016.09.003",

language = "English",

volume = "10",

pages = "1055--1078",

journal = "Journal of Informetrics",

issn = "1751-1577",

publisher = "Elsevier",

number = "4",

}

TY - JOUR

T1 - A critical cluster analysis of 44 indicators of author-level performance

AU - Wildgaard, Lorna Elizabeth

PY - 2016/11/1

Y1 - 2016/11/1

N2 - This paper explores a 7-stage cluster methodology as a process to identify appropriate indicators for evaluation of individual researchers at a disciplinary and seniority level. Publication and citation data for 741 researchers from 4 disciplines was collected in Web of Science. Forty-four indicators of individual researcher performance were computed using the data. The clustering solution was supported by continued reference to the researcher’s curriculum vitae, an effect analysis and a risk analysis. Disciplinary appropriate indicators were identified and used to divide the researchers into four groups; low, middle, high and extremely high performers. Seniority-specific indicators were not identified. The practical importance of the recommended disciplinary appropriate indicators is concerning. Our study revealed several critical concerns that should be investigated in the application of statistics in research evaluation.The strength of the 7-stage cluster methodology is that it makes clear that in the evaluation of individual researchers, statistics cannot stand alone. The methodology is reliant on contextual information to verify the bibliometric values and cluster solution. It is important to do studies that investigate the usefulness of statistical evaluation methodologies to help us as a community learn more about the appropriateness of particular bibliometric indicators in the analysis of different researcher profiles.

AB - This paper explores a 7-stage cluster methodology as a process to identify appropriate indicators for evaluation of individual researchers at a disciplinary and seniority level. Publication and citation data for 741 researchers from 4 disciplines was collected in Web of Science. Forty-four indicators of individual researcher performance were computed using the data. The clustering solution was supported by continued reference to the researcher’s curriculum vitae, an effect analysis and a risk analysis. Disciplinary appropriate indicators were identified and used to divide the researchers into four groups; low, middle, high and extremely high performers. Seniority-specific indicators were not identified. The practical importance of the recommended disciplinary appropriate indicators is concerning. Our study revealed several critical concerns that should be investigated in the application of statistics in research evaluation.The strength of the 7-stage cluster methodology is that it makes clear that in the evaluation of individual researchers, statistics cannot stand alone. The methodology is reliant on contextual information to verify the bibliometric values and cluster solution. It is important to do studies that investigate the usefulness of statistical evaluation methodologies to help us as a community learn more about the appropriateness of particular bibliometric indicators in the analysis of different researcher profiles.

UR - http://www.sciencedirect.com/science/article/pii/S1751157716301171

U2 - 10.1016/j.joi.2016.09.003

DO - 10.1016/j.joi.2016.09.003

M3 - Journal article

SN - 1751-1577

VL - 10

SP - 1055

EP - 1078

JO - Journal of Informetrics

JF - Journal of Informetrics

IS - 4

ER -