Artículo

Why summing up bibliometric indicators does not justify a composite indicator

Resumen

Various bibliometric indicators have been used to assess the researchers’ impact, but composites of such indicators, namely a metric that combines various individual indicators to describe a complex construct, have received a strong critique thus far. We employ concepts from psychometrics to revisit a composite proposed by Ioannidis et al. (2020) that aimed to represent researcher impact. Based on a selected sample of highly cited researchers, our proof-of-concept study presents a psychometrically principled composite formation. Specifically, by relying on the congeneric measurement model (and related models) rooted in classical test theory, we found that one of the proposed indicators clearly violated the congeneric model’s fundamental assumption of unidimensionality, and two other indicators were excluded for redundancy. The resulting composite based on only three bibliometric indicators was found to display excellent reliability. Importantly, the reliability approached that of the composite based on five indicators, and it was clearly better than the original six-indicator composite. Further, we found rather homogeneous effective weights (i.e., relative contributions of each indicator to composite variance) for simple sum scores, and these weights were close to those calculated using an algorithm for equally effective weights. While the congeneric measurement model also showed strong measurement invariance across sexes, this model’s loadings and intercepts were not measurement invariant across scientific fields and academic age groups. Notably, we found that various derived composites correlate positively with academic age, hinting at a lack of fairness of the composites.
https://doi.org/10.1007/s11192-024-05194-x
Why summing up bibliometric indicators does not justify a composite indicator
2024
hybrid
https://doi.org/10.1007/s11192-024-05194-x
Boris Forthmann; Philipp Doebler; Rüdiger Mutz
Institute of Psychology, University of Münster, Münster, Germany; Department of Statistics, TU Dortmund University, Dortmund, Germany; Center of Higher Education and Science Studies, University of Zurich, Zurich, Switzerland
Artículo obtenido de:
OpenAlex
Fuente
0 0 votos
Califica el artículo
Subscribirse
Notificación de
Campus Samborondón
Vía Samborondón Km. 13.5
Call Center: 5150400
metricas@ecotec.edu.ec
Facebook Instagram Youtube

Mapa del sitio