Evolving and strengthening Knowledge Exchange metrics

The Knowledge Exchange Framework (KEF)  is coming to an important point in its evolution with the review of the first iteration, and now is a hugely exciting time for me to be taking on my new role as the Head of Data and Evidence in Knowledge Exchange. Building on a very strong basis in knowledge exchange policy in roles across UKRI, I am looking forward to taking forward the KEF and evolving the way data informs KE policy at Research England (RE).

First and foremost, I am concerned about getting the metrics right. And that’s both in relation to those used specifically in the KEF but also those underpinning and supporting RE’s authoritative role in metrics more generally, especially when considering institutional comparisons. Putting care into developing meaningful and reliable indicators that are fit for purpose has always been and will continue to be essential in understanding current practices of the sector.

Closely analysing the metrics and methods used in the KEF is a critical part of our review of the first iteration. When designing and evolving the KEF, identifying and selecting appropriate metrics has been integral to accurately portraying the variety and volume of activity the sector conducts, and the review has shown that although this is by no means perfect – we are getting there. But this has only been part of the challenge, another aspect has been ensuring that the technical methods we are employing in the KEF allow any performance in any aspect of KE to shine through and be fairly represented.

A dedicated KEF Metrics Expert Group has been unpicking the nuances of the current KEF methodology. The analytical expertise and experience of knowledge exchange (and its measurement) held by this group has allowed us to investigate how characteristics of the data and methods could be affecting the KEF results of providers. Firstly, although diversity of activity across the sector is one of KE’s strengths, it also presents a significant challenge when designing appropriate indicators and manipulating data into metrics. Working closely with this group has uncovered some of the effects the distinct characteristics of the KEF dataset, such as significant outliers and compressed ranges, is having on how results of whole sector are being portrayed. I am very grateful to Tomas Coates Ulrichsen, Zoi Roupakia, Maggie Smart, Lotte Boon, Shirley Coleman, and Lyuba Dimitrova, for their time and invaluable contributions as part of this group.

Looking now beyond the KEF, the evolution of metrics across KE and the tools they provide to enable thoughtful comparisons is hugely important, and this stems from data collection. More reliable data allows more robust metrics and this means more meaningful and useful comparisons – at both an institutional and international level. My work so far at RE has allowed me to begin to delve into comparisons of IP activity across the UK and internationally. Although this provides an indication of where we are at, it is clear that there are pieces of the picture missing if we want to really understand and continue to progress practice across the UK sector. And this comes down to more meaningful and diverse data.

The latest State of the Relationship report published last week by NCUB is an example of how data and metrics to build a clear narrative of the wider activities of the sector, and is an extremely welcome and valuable resource to both the sector and to us at RE. It surfaces KE data and evidence and demonstrates the continued value of university-business collaborations.

If we are to achieve success when evolving and strengthening the metrics that underpin much of our understanding of KE policy and practice, we must continue to engage with providers, businesses and other stakeholders to ensure we getting this data right for all. A complete picture can only be built with input from the full range of relevant actors. I think that insights from stakeholders is going to be crucial when comparing practices internationally, an area where I believe there to be a lot of potential. I look forward to working closely with providers as this work progresses.