23 January 2025 – Researchers from the Global Young Academy (GYA) Scientific Excellence working group have published an article in the journal Nature examining the criteria used to measure success in academia.
Having carried out a first truly global analysis of 532 policies adopted by institutions or government agencies from 121 countries on the promotion to the post of (full) professor, the authors found that publication and citation metrics are not uncommon, but surprisingly, policies affecting approximately three-quarters of researchers around the globe do not contain an explicit reference to citations. Rather, mentoring, administrative responsibilities, as well as contributions to the field or scientific community frequently play a role.
Unlike previous studies that focused on the prominence of criteria, the findings from this publication result from an approach that looked for both commonalities and differences, as well as the clustering of criteria within given policies.
This approach revealed four characteristic profiles of assessment – those considering (1) output metrics, (2) visibility & engagement, (3) career development, and (4) outcomes & impact.
GYA alumnus Martin Dominik (University of St Andrews, United Kingdom) an initiator of the study, points out that "Research institutions around the world are not identical and interchangeable, criteria differ, and anyone interested in a job in academia should be well aware of specificities."
The pervasiveness of evaluation metrics means that universities are under pressure by rankings that set their own criteria. Still, some adopted assessment criteria are quite contrary to what an institution wants to achieve and some particularly reward bad practices, including fraud and anti-social behavior within the research community.
Many differences were found between countries, with simple publication and citation metrics being particularly popular in upper middle-income countries, aspiring to close the gap to stronger economies. Such metrics have a clear appeal due to providing an easy and straightforward way to measure progress or success. But these lose purpose on failing to align with goals that adequately serve society in the specific environment.
Moreover, many popular metrics come with systemic disadvantages for less-developed countries and researchers working there. Focusing on those as measures of success results in engaging in a competition that cannot be won and staying behind forever, rather than embracing opportunities based on specific strengths that enable leaping ahead.
Building assessment around normative career paths ignores that candidates faced different challenges under different circumstances and within different environments. Such an approach also creates substantial obstacles to inter-sectoral mobility across academia, industry, government, and not-for-profit organisations.
Evaluation practices that are shaped by a universal "excellence" narrative not only fail to support the specific goals of institutions, but moreover restrict academic diversity and reinforce Global North-South inequalities.
GYA Co-Chair Yensi Flores Bueso (University College Cork/University of Washington, Ireland/United States), concludes "I hope that this work provides a foundation to rethink policies so that they foster equity, inclusivity, and research integrity as fundamental pillars of our research culture, and that consequently science can serve as a common good for humanity as a whole."
Read the Nature article here: " Regional and institutional trends in assessment for academic promotion ".
