Challenges with measures used for assessing research impact in higher education institutions

Abstract

Internationally, there has been a push for the prioritisation of research impact beyond its scholarly contribution. Traditionally, research impact assessments have focused on academic impact and quantitative measures, at the expense of researchers for whom research impact cannot be quantified. Bibliometric indicators and other quantitative measures are still the most widely used method for evaluating research impact because these measures are easy to use and provide a quick solution for evaluators. Conversely, metric indicators fail to capture important dimensions of high-quality research. Hence, in this study, we explored challenges with metric indicators. We adopted a case study of the University of Cape Town and used document analysis, a questionnaire survey to collect data from academics and researchers, as well as semi-structured interviews with a sample of academic and research staff. The findings highlight common challenges with quantitative measures, such as bias and discipline coverage, and the ability of measures to drive researchers’ behaviour in another direction. We propose the adoption of responsible research metrics and assessment in South African higher education institutions for more inclusive and equitable research impact assessments.

Significance:

• The study highlights the importance of understanding the challenges and influence of current measures used for assessing research impact in higher education institutions.
• There is a need for higher education leaders, policymakers and funders to advocate and support responsible metrics.
• Higher education leaders, funders and policymakers need to collaborate at the national level to initiate and support research assessment reform.