Artículo

Why summing up bibliometric indicators does not justify a composite indicator

Resumen

Various bibliometric indicators have been used to assess the researchers’ impact, but composites of such indicators, namely a metric that combines various individual indicators to describe a complex construct, have received a strong critique thus far. We employ concepts from psychometrics to revisit a composite proposed by Ioannidis et al. (2020) that aimed to represent researcher impact. Based on a selected sample of highly cited researchers, our proof-of-concept study presents a psychometrically principled composite formation. Specifically, by relying on the congeneric measurement model (and related models) rooted in classical test theory, we found that one of the proposed indicators clearly violated the congeneric model’s fundamental assumption of unidimensionality, and two other indicators were excluded for redundancy. The resulting composite based on only three bibliometric indicators was found to display excellent reliability. Importantly, the reliability approached that of the composite based on five indicators, and it was clearly better than the original six-indicator composite. Further, we found rather homogeneous effective weights (i.e., relative contributions of each indicator to composite variance) for simple sum scores, and these weights were close to those calculated using an algorithm for equally effective weights. While the congeneric measurement model also showed strong measurement invariance across sexes, this model’s loadings and intercepts were not measurement invariant across scientific fields and academic age groups. Notably, we found that various derived composites correlate positively with academic age, hinting at a lack of fairness of the composites.
Mariappan, Punithavili (58284795700); Khairani, Mohd Zahuri (57201092990); Mohd, Norzuraina (58922805700); Chanthiran, Maran (58284611900); Cubalit, Andy Noces (58733882200)
Uncovering Emerging Trends in Technology and Art Education: A Bibliometric Mapping Analysis
2024
10.37934/araset.41.1.6475
https://www.scopus.com/inward/record.uri?eid=2-s2.0-85186909464&doi=10.37934%2faraset.41.1.6475&partnerID=40&md5=3ef9fa96ecc9ca3f2339b253cf7ebee3
Faculty of Sustainability Arts and Creative Industry, Universiti Pendidikan Sultan Idris, , Perak, Tanjong Malim, 35900, Malaysia; Faculty of Computing and Meta-Technology, Universiti Pendidikan Sultan Idris, Perak, Tanjong Malim, 35900, Malaysia; King Mongkut’s Institute of Technology Latkarabang, Bangkok, 10520, Thailand
All Open Access; Hybrid Gold Open Access
Scopus
Artículo obtenido de:
Scopus
Fuente
0 0 votos
Califica el artículo
Campus Samborondón
Vía Samborondón Km. 13.5
Call Center: 5150400
metricas@ecotec.edu.ec
Facebook Instagram Youtube

Mapa del sitio