Artículo

Why summing up bibliometric indicators does not justify a composite indicator

Resumen

Various bibliometric indicators have been used to assess the researchers’ impact, but composites of such indicators, namely a metric that combines various individual indicators to describe a complex construct, have received a strong critique thus far. We employ concepts from psychometrics to revisit a composite proposed by Ioannidis et al. (2020) that aimed to represent researcher impact. Based on a selected sample of highly cited researchers, our proof-of-concept study presents a psychometrically principled composite formation. Specifically, by relying on the congeneric measurement model (and related models) rooted in classical test theory, we found that one of the proposed indicators clearly violated the congeneric model’s fundamental assumption of unidimensionality, and two other indicators were excluded for redundancy. The resulting composite based on only three bibliometric indicators was found to display excellent reliability. Importantly, the reliability approached that of the composite based on five indicators, and it was clearly better than the original six-indicator composite. Further, we found rather homogeneous effective weights (i.e., relative contributions of each indicator to composite variance) for simple sum scores, and these weights were close to those calculated using an algorithm for equally effective weights. While the congeneric measurement model also showed strong measurement invariance across sexes, this model’s loadings and intercepts were not measurement invariant across scientific fields and academic age groups. Notably, we found that various derived composites correlate positively with academic age, hinting at a lack of fairness of the composites.
Autores
Pedraja-Rejas, L; Rodriguez-Ponce, E; Munoz-Fritis, C; Laroze, D
Título
Sustainable Development Goals and Education: A Bibliometric Review-The Case of Latin America
Afiliaciones
Universidad de Tarapaca; Universidad de Tarapaca
Año
2023
DOI
10.3390/su15129833
Tipo de acceso abierto
gold
Referencia
WOS:001015693100001
Artículo obtenido de:
WOS
Fuente
0 0 votos
Califica el artículo
Subscribirse
Notificación de
Campus Samborondón
Vía Samborondón Km. 13.5
Call Center: 5150400
metricas@ecotec.edu.ec
Facebook Instagram Youtube

Mapa del sitio