In this blog, we analyse ‘pathways to impact’ in grant applications that led to the top scoring impact case studies in the latest UK assessment of research excellence and impact (REF2014). It is easy to find the top scoring case studies, and straight-forward enough to find good examples of pathways to impact, but this is the first time that pathways have been paired with high-scoring case studies.
To do this, we contacted those responsible for the majority of impact case studies we could identify that scored the highest grade (4*) in REF2014, and asked if they would allow us to share their pathways to impact. In our analysis, we have identified elements that pathways to winning impact case studies have in common, from a wide range of disciplinary areas. In doing so, we provide suggestions that complement official guidance on writing ‘pathways to impact’ and can help you develop a pathway to significant and far-reaching impacts.
A bit of context
Countries around the world are developing systems for evaluating the excellence and impact of Government funded research, and the UK is ahead of most in its assessment of research impact. The 2014 Research Excellence Framework requested, for the first time, that institutions evidence how some of their research had delivered beneficial impacts for "the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia". Research Councils UK (RCUK) “encourage researchers to be actively involved in thinking about how they will achieve excellence with impact and to explore pathways for realising the impact of their research”. RCUK continue to work with the research community to develop its guidance for Pathways to Impact and you can read their guidance, including FAQ’s and top tips, for Pathways to Impact here.
The evidence
At the beginning of 2015, HEFCE published a searchable database of impact case studies, collected as part of its evaluation of UK research under their REF. We used this database in a blog we wrote on top ways researchers achieve policy impacts and decided to make use of it again for this analysis.
Sarah reached out to 50 researchers from a range of institutions and disciplines that had achieved 4* impact case studies, and we selected six of the pathways to impact we received for analysis. For comparison, we have also included the pathway to impact for a lower scored (3*) case study that Mark wrote. In addition to a detailed thematic analysis of these documents, we gathered general information from the researchers who didn’t have any impact planning documents to share. Most of the successful impact case studies were based on grant applications that were written prior to the introduction of pathways to impact sections, but instead they sent us the most relevant part of their application or other impact planning documents they had developed at the start of the research (Table 1).
Table 1: Impact case studies used for the analysis, showing the nature of impact planning documents provided, the score given under the Research Excellence Framework (REF), the submitting institution and the discipline (REF Unit of Assessment)
Pathway | Case Study Implementing Evidence-based Community Stroke Services
Pathway | Case Study Displaying the Flag: Transforming Conflict in Northern Ireland
Pathway | Case Study Preventing Psychosocial Risks and Work-Related Stress in Europe: Impact on Policy and Practice
Pathway | Case Study Changing Policy And Practice In The Prevention Of Suicide And Self-Harm
Pathway | Case Study Mapping Medieval Chester: driving heritage policy, expanding heritage audiences and creating new cultural and economic opportunities
Pathway | Case Study The Old Bailey Online: Democratising access to social history
Pathway | Case Study Paying for Nature’s Services: Developing the UK Peatland Code
10 lessons from pathways to top-rated impacts
Although impact is now a focus for research funders around the world, guidelines are often vague and there are few examples of good practice available from which to generalise lessons. Here we have identified ten lessons from pathways to top-scoring impact case studies:
1. Clear connectivity from overall vision to objectives and impact
In all the highly rated case studies, the impact planning demonstrated a strong connection from the overall research vision and purpose to the impact objectives and outputs. The take-home message is to make sure there is a strong narrative – a thread running through the whole programme of work. Illustrating this in some way can be helpful to get your team on board and see how everything fits into the bigger research picture. Take a look at this Goals to Impact document provided by Dr Fisher at the University of Nottingham.
2. Specificity
This seems like an obvious point but when you compare successful pathways to impact with those that failed to get such a good rating, you really start to notice just how important it is to BE SPECIFIC. It’s very easy to get caught up in answering ‘what’ you will be doing where in actual fact that can usually be described in a few sentences, e.g. “we will develop an online learning platform.” Overall, the real crux of impact information will fall into three categories; the ‘(audiences), the ‘ (methods) and the ‘(dates, phases).
The ‘who’ - Identifying specific audiences.
All of the successful REF impact case studies identified very specific target audiences. It is not necessary to cover a wide breadth of people but demonstrating you have carried out an audience analysis and will be targeting the most relevant groups or individuals makes your impact more targeted and more likely to be successful. Some detailed plans even listed individual names and numbers of individuals within certain target groups. This may not be necessary in all cases but the level of detail is important here. Stating ‘general public’ may be suitable in some situations, but detailing ’we will make contact with 14 members of X institute’ is more tangible. The Pathway to Impact from the University of Hertfordshire planned to create an open-access, on-line and searchable edition of the Old Bailey Proceedings, 1670-1834. In the Pathway to Impact, three specific user-groups were identified and the document stated that “for each of these groups we have undertaken an extensive, but non-statistical, survey of need, based on the distribution of a project proposal and a request for direct (but unstructured) feedback.” The research on the audiences had already been carried out so readers of the pathway knew that the research team were completely ready to go. They had also detailed how many organisations within each user group they had already consulted with. When compared to the groups identified in Mark's 3* case study (e.g. government departments and future buyers), there is a lack of detail that makes the pathway less convincing.
The ‘how’ – how will you impact these groups?
If you’ve gone into detail as to who you are targeting with your impact, then the methods of how you will achieve this need to be just as detailed. For example, if you’ve identified 15-25 year old environmental activists as your target audience for impact, you then need to detail exactly how you will engage with them. This is the point to draw on specific team members that have existing relationships, e.g. “Professor Carle has been working with Activist Group A for four years and will lead a workshop with 15 of his contacts at Organisation X”.
The ‘when’ – plan your impact in clear, logical phases All of the pathways to impact that achieved great results (and high ratings) followed logical phases when planning impact. You need to make your plan logical in terms of when and where you are planning impact. Implementation for one of the 4* impact case studies approached services across specific geographical areas during certain time-periods. Others divided their impact into clear thematic areas. However they did it, all of the successful pathways had clear ‘phases of impact’.
3. Tailor-made impact
This links back to the specificity and the research behind your target audiences. The 3* pathway discussed holding meetings and producing policy briefs. The 4* pathways detailed what type of meetings (one-on-one, small group, online, in-person), where they’d be held, who would attend – and they did this for each target audience showing that their events were tailor-made for each target market. They detailed the themes and subjects of policy briefings, as well as listing the specific team or individual it would be sent to. You need to take the time to identify and understand your audiences before you can do this, but if you’ve done that then get creative and tailor-make your impact.
4. Build in flexibility
Although pathways were specific, they also built in flexibility. However, they all demonstrated that having a flexible plan doesn’t mean it has to be evasive. You can still be specific within certain phases of the research, for example collecting evidence in early stages of the research, and then building knowledge exchange and impact around that evidence as it arises. As one impact plan stated, “These activities will be tailored to the stakeholders needs and requirements and will build on evidence collected in earlier parts 2 and 3 of the study”.
5. Assign responsibility – name names
All of the successful pathways assigned research team members to specific impact tasks (including monitoring impact) – giving them responsibilities. Seeing specific names on the pathway can often legitimise the work because it shows how people are taking responsibility. Dr Leka at University of Nottingham advises to assign team members to impact monitoring. In the related impact plan her team states that, “a specific dissemination team is incorporated into the management structure of the project.”
6. Demonstrate demand
Where possible, collect evidence (e.g. market research) that there is a real and pressing need for the impacts you are seeking to generate. For example, if you are creating a web resource, get statistics for the number of visitors to similar types of website. Case study number 6 detailed enquiry statistics as well as general website usage figures relevant to the resource they were creating. Not only had they identified their audience – they also identified a need for their impact activities, evidenced by facts and figures.
7. Highlight collaborative partnerships
The successful pathways named specific partnerships that they either already had (and were planning to capitalise on) or that they didn’t yet have (but planned to develop). Again, name names and demonstrate an impact plan that is aware of its environment. Develop activities that make use of who and what is already working in the area. If you can collaborate with a partner to create a larger event, then do it. If you can co-produce a text-book with a significant organisation then do it. Partnerships can significantly increase your level of impact, and sometimes decrease the level of staff time.
8. Don’t ignore sensitivities
Rather than trying to avoid mentioning any sensitive issues, highlight them to demonstrate the background (market) research and other ground work you have carried out. Mention ethics protocols that will be followed and organisations you will work with to ensure you communicate sensitively with vulnerable groups. For example, Professor O’Connor and his team noted in their NHS impact planning documents that their research was of a sensitive topic and that they would “ensure that the dissemination is carefully organised and consistent with the International Association of Suicide Prevention’s guidance on reporting on suicide (and related issues) in the media.” Mention any relevant bodies and be specific – write names in full and provide a brief explanation if the name isn’t self-explanatory.
9. Think long-term
Demonstrate how you are thinking long-term with regards to impact. What will happen at the end of your funded research? Can you describe how what you are putting in place is self-running? Or that you will inspire a group of people who will then manage a process or output?
10. Record everything
This final point isn’t specifically for the impact planning, but rather as good practice for every stage of the impact process. All of the 4* REF Impact Case Studies had something in common – they had been mindful about capturing impacts throughout the lifetime of the study. For example, Professor Land at Durham University found a large archive maintained by his colleague Dr Mick Flanagan at UCL “invaluable” when it came to submitting their REF Impact Case Study. He would recommend researchers start constructing something similar to capture their own work as, although Dr Flanagan set it up before impact studies were being discussed, “it turned out to be an ideal vehicle for demonstrating and recording impact for the REF”.
Conclusion
Often impact is the result of an entire career of work and we received many responses explaining that an impact planning document didn’t exist, as it was just carried out as part of the work before RCUK wanted these documents. As Dr Fisher at the University of Nottingham stated; “For me, the whole study is part of a pathway to impact – rather than say a discrete set of activities at the end of a clinical trial.”
An article written by Neil Jarman and Dominic Bryan discusses their experiences of demonstrating impact as part of a UK anthropology department. They start the article by saying that their department has “long had a commitment to engaging with policy and practice and applying their research to inform communities and government in Northern Ireland.” These first few lines hold the key to successful impact. While carrying out this analysis of REF Impact Case Studies, the one factor that all successful case studies appear to have in common is passion and commitment to the research. The top-scoring case studies we reviewed were written by researchers who were passionate, who cared about what they were doing and so, as a logical extension, they wanted their work to have an impact.
There is no ‘right way’ to generate impact and the documents that researchers shared with us show the diverse range of approaches you can take to planning for impact as part of a research proposal. Nevertheless, there are a number of generalisable lessons that emerge from our analysis of the pathways that led to top-scoring impacts, and we hope that they inspire you to write your next pathway to impact with creative flare and confidence. A credible pathway to impact inspires confidence in reviewers and funding panels that you will actually be able to generate the impacts you claim. The more you invest in creating a credible pathway to impact prior to submiting your bid, the easier it then becomes to turn this into a full impact plan for your funded project (check out our free impact planning template and worked example if you've not seen this yet, in the resources section of our website). If you've got a pathway to impact that you'd like to share with other researchers via this blog (whether or not you can link it to a 4* impact case study), please get in touch.
Many thanks to the researchers who shared their pathways to impact, to make this blog possible.