REF: So far, so good

This article originally appeared in Research Fortnight.

Last month’s results of the Research Excellence Framework (REF) released a wave of instant analysis. Conclusions were relayed with breathless excitement. Universities claimed to be top of this or fastest-growing that.

“Impact case studies have helped to bury the myth that the economic pay-off from research comes mainly through applied research and spinout companies.”

Vital questions required fast responses: what do the results mean for the funding councils, regions, metrics, international comparisons, business collaboration, research disciplines, careers? Eminent scholars were calling for the process to be reviewed even before the results were published. Phew.

The Christmas break brought a chance to reflect on the outcomes. Power—the volume of work submitted multiplied by the quality of those submissions—emerged as the metric of choice, both for analysts and the national press. Most commentators appeared to accept the results as a fair assessment of research in UK institutions. As far as the appraisal of research quality goes, that’s no surprise—we have had 30 years of practice.

Impact is the new game in town and it is a testament to all those who took part in the REF that impact assessment worked so well. Of course it is not perfect—how could it be? But it is a pretty good first attempt.

The arts, humanities and social sciences are among the winners. Academics from these fields seemed to spend half their time saying that impact assessment couldn’t be done and the other half doing it splendidly.

Perhaps some disciplines had previously lacked the incentive or opportunity to describe systematically their contributions to society and the economy. Single-subject institutions, many of them in the arts, are clustered at the top of the grade point average charts on impact. The very purpose of these institutions is to deliver impact. Research assessment has caught up with what they have been doing for decades.

“It is the combined strength of many institutions and a business-friendly culture in UK academe that helps make this country the most attractive in the G7 for inward R&D investment.”

The major research universities also did well, and now have the highest scores for impact power added to their strength in research. This confirms that large bodies of world-leading research deliver the greatest impact. Smaller bodies of excellence are also on display at a range of institutions.

The correlation between research excellence and high impact endorses the policies of successive governments to fund excellence and resist the temptation to distort scarce research funding to serve a regional agenda. In any case, the impact of research reaches far beyond where the work is done. Take a few examples from my own institution: improved treatment of tuberculosis; better street lighting; improved pre-school provision; and fighting organised crime. The research was done in London but the impact spreads across the UK and beyond.

Impact case studies have helped to bury the myth that the economic pay-off from research comes mainly through applied research and spinout companies. But these are small parts of a much larger picture.

The REF has shone a light on the unpredictable, chaotic, exciting paths that knowledge travels as it makes its way to impact. It has shown the intrinsic value of expanding the frontiers of knowledge.

Some extraordinarily talented researchers pursue knowledge rather than impact. They expand and enrich the ecosystem of research, from which impact is harvested at other times and places. Without the new knowledge, the ecosystem would wither.

“Let’s focus on reaping the benefits of the REF—and particularly the impact agenda”

This takes us to the limits of what the REF can capture. For example, the UK wins an unusually large share of global firms’ foreign direct investment in R&D. This adds several billion pounds each year directly to the UK’s GDP. There is clear evidence that the excellence of the UK research base is a major factor in attracting this investment. This excellence is a national characteristic; REF results come from individual researchers and departments.

The largest research universities may have sufficient global visibility to attract investment on their own. But generally it is the combined strength of many institutions and a business-friendly culture in UK academe that helps make this country the most attractive in the G7, the most powerful economies in the world, for inward R&D investment. This is a dimension of impact that the REF may never capture.

What happens next? The Higher Education Funding Council for England already has an evaluation under way. Let’s focus on reaping the benefits of the REF—and particularly the impact agenda—before too many other reviews.

We have passed the difficult stage of writing the first-ever impact case studies. The impact agenda now needs time to mature. Metrics may well streamline the assessment of research quality, which would be welcome. But surely we should leave impact alone.

Graeme Reid is NCUB strategic adviser to NCUB.