Enhancing your Impact

Getting to the root of negative attitudes towards research impact

Philosopher and Fast Track Impact trainer, Dr Jenn Chubb, recently completed her PhD, which provides valuable new insights into academic perceptions of research impact. Her thesis, titled Instrumentalism and Epistemic Responsibility: Researchers and the Impact Agenda in the UK and Australia, sheds light for the first time on the depth of negative emotion provoked by the impact agenda on both sides of the world, linked to its political roots, the way incentives have been designed and evaluations are managed. In this article, we share some of the key insights emerging from her work, and unpack both the fear and hope expressed by those she interviewed.

Deeply felt concerns

Researchers expressed concerns that ranged from the perception that impact was “strangulating” creativity, to concerns about negative e ects on career progression for those who could not demonstrate impact from their work. Dr Chubb’s research shows how the political roots of the impact agenda (as a way of justifying government investment in research) have led to knowledge becoming increasingly politicised and marketised, resulting in over-regulation and managerialism in the academy. This has fuelled ongoing controversy in a community which is heavily emotionally invested in its knowledge.

The thesis shows how this has had negative impacts on academic identity and behaviour, leading to claims by some academics that they feel the need to “sensationalise” and “embellish” impact narratives to get funding. The research raises fundamental questions about how intellectuals and their knowledge interact with society. It unearths deeply entrenched ideals about how we value knowledge; who it belongs to and what it should do for whom. As a result, the types of knowledge traditionally created through research are changing, with some researchers feeling increasingly under threat.

A new disciplinary hierarchy?

Tensions were felt more by certain people than others, depending on their motives for doing research and their disciplinary background. For example, curiosity-driven researchers may feel the impact agenda is an attack on their academic and even personal identity, trying to turn them into someone that they are not. Researchers from life and earth sciences and social science were more likely to conduct instrumental research of extrinsic value, and perceive impact as an opportunity. At the opposite end of the spectrum were researchers from the arts and humanities and physical sciences doing less instrumental research of more intrinsic value, who were more likely to be ambivalent about impact or see it as a threat. Although there are clearly still ingrained perceptions of academic hierarchies, favouring the intrinsic value of pure research over applied research and practice, Dr Chubb’s research suggests that the impact agenda may be turning this hierarchy on its head.

Duty versus accountability

Academics who were interviewed typically felt torn between protecting academic freedom versus their “academic duty” to communicate their research and benefit the publics who funded their work. However, insights emerge from Dr Chubb’s work that challenge this tension with the idea of “accountable academic freedom”. Instead of seeing impact as a duty to be discharged, accountability is a positive moral imperative that competes with the idea of unaccountable academic freedom. If impact is seen as making academic freedom accountable, then it is no longer a trade-o that has to be made, but rather it justifies and deepens academic freedom.

Find out more

Jenn Chubb was awarded her PhD at the University of York. Her background is in philosophy and she has a particular interest in virtue ethics, academic freedom, the instrumentalism of knowledge and epistemic responsibility. She does impact training for Fast Track Impact and is a freelance trainer specialising in grant writing. She has published four papers from he thesis so far, and more are on the way:

Chubb, J., Watermeyer, R. and Wakeling, P. (2017) Fear and loathing in the academy? The role of emotion in response to an impact agenda in the UK and Australia. Higher Education Research and Development 36: 555-568
Chubb, J. and Watermeyer, R. (2016) Artifice or integrity in the marketization of research impact?: Investigating the moral economy of (pathway to) impact statements within research funding proposals in the UK and Australia. Studies in Higher Education 1-13
Chubb J, Reed MS (2018). The politics of research impact: implications for research funding, motivation and quality. British Politics

Also, she has written about her work in The Conversation:

Academics fear the value of knowledge for its own sake is diminishing: https://theconversation.com/academics-fear-the-value-of-knowledge-for-its-own-sake-is-diminishing-75341.

Academics admit feeling pressure to embellish possible impact of research: https://theconversation.com/academics-admit-feeling-pressure-to-embellish-possible-impact-of-research-56059

Two ways to set powerful impact goals

The weakest part of many impact summaries and pathways to impact in grant applications is their impact goals. Goals are often hidden in dense introductory paragraphs, they may be vague, and in some cases they are completely missing. Part of the reason for this is that it is hard to clearly state convincing impact goals. Researchers are trained how to come up with research goals, not impact goals. No matter how hard it may be, this is a skill that researchers increasingly need if they want to get their work funded. Here are two techniques that make the process of setting impact goals easier.

The ‘5 whys’ approach

This method does exactly what it says on the box. You simply keep asking yourself “but why does that matter?” Try and articulate what you think is interesting about your work, and then ask why that matters and to whom does it matter. Take your answer and ask yourself the question again: “but why does that matter”? To be most effective, run this as a discussion with a colleague asking you the question. You may not get to the fifth “why”, but you are likely to be significantly clearer than when you started.

The logic model approach

Logic models usually start by identifying your goal and working back from it to identify activities that will enable you to reach your goal. This isn’t much help if you’re struggling to identify clear goals. However, you can use the Fast Track Impact Planning Template (a type of logic model) to help you identify your goals:

Instead of starting in this first column with your impact goals, start in the second column, identifying people or groups outside the academy who might be interested in your research, and the aspects of your research that you think they would be most interested in.
Ask yourself how this person or group would benefit as a result of satisfying that interest, and you will be able to start identifying a prototypical impact goal.
Next, start to identify activities for each group of people that has different interests in your research, which would help them benefit from your work in some way.
Then identify indicators that would tell you if the activities were successful or not, and whether they delivered benefits to the people you engaged with. To come up with impact indicators, close your eyes, and imagine yourself standing years from now after you have completed your research. What do you see? What are people telling you has happened?
Now, work out how you would be able to prove that those changes had happened. What data would you collect?

Use the insights you gain from this process to revisit your prototypical goals and make them more specific, measurable and achievable with the resources and time you have.

Download the Fast Track Impact Planning Template at: www.fasttrackimpact.com/resources

Engaging with business

Engaging with business for impact versus income

Commercialising research is fraught with challenges, and now the impact agenda has added a new complication. In many cases, commercialisation is a win-win situation, providing income to you and your institution, whilst providing economic benefits to the companies you work with and many wider benefits to their customers. However, researchers can be faced with a challenging trade-o if the companies they work with want to bring products to market at high price points, keeping profits high but limiting the number of people who are able to benefit from the research, and so limiting impact. High prices may reflect the investment a company needs to make to bring the product successfully to market, and the risk they perceive of not making a return on this investment. In some cases, however, this can be disastrous for impact, for example, if a new product is unavailable to all but a handful of consumers, companies or countries that can afford to buy it.

Setting up a spin-out company to bring your product to market is one way of retaining some level of control over pricing, and you can if you wish establish your spin-out company as a not-for-profit to keep prices as low as possible and get your product into as many hands as possible. However, for many types of business and product, your lack of business expertise and experience will significantly limit your routes to market and so limit your capacity for impact.

The alternative is to develop a negotiating strategy with colleagues from your institution who specialise in IP, based on maximizing impact instead of income. Whether going down the route of a sole license or licensing to multiple companies, if you have a well-developed product that you believe in, you can negotiate for impact instead of income, if that is what is most important to you. For example, you may limit the royalties you receive in return for commitments over pricing to ensure wide and equitable access to the outcomes of your research.

Most research institutions seek to maximise income from their IP. In most cases, this is a win-win for income and impact, but where there are potential trade-o s between income and impact, you may need to be the one who sees the bigger picture, and proposes a different negotiating strategy.

How to write a winning pathway to impact and impact summary

by Mark Reed

Many researchers find the impact sections of their grant applications among the most challenging to complete. My new guide explains exactly what you need to write in the two separate impact sections in a Research Council bid (your impact summary and your pathway to impact), and also applies to the impact sections of grant applications for other funders.

A strong impact summary and pathway to impact can make the di erence between getting funded or not if your application is tied with others in the “danger zone” near the funding cut-o . Being able to demonstrate impact is even more important if you are applying for funding for the Global Challenges Research Fund, where you have to demonstrate how your work will contribute to Official Development Assistance.

You can see best practice examples of impact summaries and pathways to impact on our website: http://www.fasttrackimpact.com/ pathways-to-impact. If you have a good example, get in touch—the more examples we receive, the more useful this resource will be.

What should be in my impact summary?

The impact summary is meant to answer just two questions:

Who might benefit from this research?
How might they benefit from this research?

To answer these questions, all you need to do is to: i) Clearly articulate impact goals (not dissemination or knowledge exchange goals—that’s part of your pathway to impact); and ii) list (and group) your publics and/or stakeholders. The next two sections explain how...

How can I identify powerful impact goals?

Start by identifying clear impact goals, if possible making them as specific, measurable, achievable, realistic and time-bound (SMART) as possible. Struggling? Try these tips:

Visualise yourself at the end of your project having achieved an impact that everyone is talking about. Where are you and what can you see? What has changed? What are people saying about how they have benefited?
Make sure your impact goals aren’t simply about communicating your research findings
If they are, then ask yourself who is most likely to be interested in your work outside academia, and how those who hear about your work are likely to benefit from or use what they learn
If you don’t know the answer to these questions, just focus on trying to identify the aspects of your work that you think people outside academia are most likely to be interested in. Then ask yourself why you think they might be interested in this aspect of the work
If you’re still struggling, go out and speak to some of the people you think might be interested, and ask them what interests them most, what might make it more interesting/relevant to them, and how they would like to benefit from or use your work
If you have a goal that is all about communication rather than impact, then you might have a good idea of the sorts of modes of communication you want to use (e.g. social media, film), and an alternative is to work back from the communication method you’re interested in using, to the people who will engage with that method, and then their interests and how they will benefit. Beware that in some cases you may discover that the communication method you want to use will not actually reach people who are interested or can use your work (for this reason it is always best to start with the goal and/or your publics/stakeholders first, before choosing your pathways to impact)
Download the Fast Track Impact Planning template from www.fasttrackimpact.com/ resources for a structured method of linking impact goals to publics/stakeholders, research findings and pathways to impact. If you find it hard starting with the goals, try and start by identifying your publics/stakeholders and what they might be interested in, and then work back from there to your goals.

Write a winning pathway to impact

How do I know who might benefit from my work?

Now you’ve got some clear impact goals, you need to identify the publics and/or stakeholders that will benefit when these goals have been achieved. Here are some tips to make this easy:

If you have limited knowledge and experience of publics/stakeholders working in your area, team up with a colleague who knows more. If you have time and contacts, consider inviting someone from outside academia who works with the people you want to help, and get them to advise you on the key groups you need to reach out to
For stakeholders, consider the relative interest each group or organisation has in your work, and their relative influence over your ability to achieve your impact goals. This influence could be negative (blocking you from achieving impact) or positive (enabling you to achieve things that would not have been possible without their help)
For publics, in addition to considering their relative interest in your work, consider the extent to which different groups (e.g. demographics, interest groups) might benefit from your work

See the graphics below for examples of actions you can take with each of the categories of publics and stakeholders that emerge from this analysis
Reach out to as many of the groups that emerge as benefiting strongly or being highly influential before you submit your grant application to get their feedback and help with your pathway to impact. This will lead to a stronger, more credible pathway and will give these groups a greater sense of joint ownership, making them more likely to engage if you get funded
Download the Fast Track Impact stakeholder and publics analysis template to do a full analysis. You won’t have room to put all of this information in your impact summary or pathway, but you will be able to use this information to group publics and stakeholders into categories (e.g. third sector, business, policy, or different sectors, socio-economic classes or interests), make strategic choices about who to highlight as key collaborators and give you a level of detail that will make your impact summary and pathway highly believable.

What are the essential things every pathway to impact should include?

According to JeS Help, the Research Councils are looking for four things:

Activities that actively engage relevant stakeholders/publics;
Activities that meet their needs, interests and priorities;
A clear plan (including "timing, personnel, skills, budget, deliverables and feasibility”; and
Your track record with stakeholder/public engagement and impact.

Here, I have tried to condense this mountain of advice down to the 10 most important things you need to make sure you don't forget:

1. Be specific

The number one piece of advice is to be specific. Tell reviewers exactly who you will work with (not just government, or even a particular department, but the specific policy team, and, if you have it, the name of your contact in that team). Specify your goals clearly, with specific indicators that will tell you when each goal has been met. Explain how you will complete each activity in credible detail and why this is the best way of achieving a specific impact e.g. instead of social media, identify the platform you will use, who you will target that is on that platform, and what impact goals you will be able to preferentially achieve via this medium.

2. Demonstrate demand or interest in your work

Find evidence of growing public interest in the issues you are studying, numbers of people attending public engagement events or watching programmes linked to your subject. Demonstrate that stakeholders want/need your work, and if possible co-develop your pathway to impact (and in some cases the whole project) in collaboration with them. Establish an advisory panel (there is actually

peer-reviewed evidence that these lead to impact more than many other pathways) and name the people you have invited, indicating where they have confirmed involvement.

3. Check you have activities to reach each of your goals

Systematically check if you have activities that will take you to each of your impact goals, and that you have identified activities that match the needs and preferences of each public/stakeholder group you identified in your impact summary.

4. Make it two-way

Where possible, focus on two-way engagement with publics and stakeholders rather than one-way communication of findings, so you get feedback and can adapt your approach to be as relevant and useful as possible. There is research evidence(iv) that projects that co-design outputs in collaboration with the people who need them, achieve greater uptake of their outputs because they are more relevant and people have a sense of shared ownership. Even for communication outputs like policy briefs, getting feedback from your target audience during the writing process can significantly increase the likelihood that your communication hits its mark.

5. Link to your impact track record

Talk about your track record on achieving impact, ideally with the groups and issues linked to your proposal. It is diffcult to “prove” that you will be able to do what you are suggesting you will do, and some of the best evidence you have is a track record of having delivered impacts for these groups in these areas in the past. If you haven’t got a track record yourself, consider bringing someone into your team who does and get them to work with you on your pathway to impact.

6. Build in impact evaluation

Have a plan for evaluating whether or not you are moving towards or away from impact, which will tell you when you have achieved your goals. The process of identifying

indicators will help you identify clearer and more credible impact goals. Thinking in detail about how you will know if you achieved impact will often identify risks and challenges that you can prepare for, making your plan even more credible. You can build in any costs of monitoring and evaluating impact into your proposal.

7. Cost it

Cost your pathway to impact and justify your request for these resources (if you are short of room in your Justification of Resources you can refer reviewers to your pathway to impact and vice versa). This shows how seriously you are taking impact, and adds credibility to your claim that these activities will actually happen. Some directed calls for proposals from the Research Councils in the past have suggested approximately 10% of the total budget should go to support Pathways to Impact. Researchers typically put in significantly less than this, fearing negative feedback from reviewers on their "value for money", but anything between 5% and 10% is reasonable.

8. Weave in impact to your research plan

If possible, weave your pathway to impact into your research plan, cross-referencing to it from your case for support at relevant points.

9. Keep it simple

Use plain English and make your pathway to impact stand alone (e.g. spelling out acronyms), as a lay member of a funding panel may only read the impact-related parts of your proposal in any detail.

10. Seek specialist impact pre-review feedback

Don't rely on academic pre-reviewers to provide feedback on the impact sections of your proposal. Instead, seek feedback from someone in your university who specialises in impact or, if possible, get feedback on these sections from someone who works with the publics or stakeholders you want to benefit.

What if I am doing pure research that will not have any impact?

It is really difficult to come up with any sort of impact for some very pure, non-applied projects. In this case, you cannot get away without producing an impact summary and pathway to impact if you want funding from the Research Councils. You don't have to use all the characters and pages you are given, but you do need to think about what the next steps might be, even if these happen many years after your research is done, that might possibly provide economic or societal benefit. You don't have to be right and no-one will hold you to this—just make some educated guesses. Do not, however, be tempted to include additional benefits for academics, students and the academy in this section, or you may risk your pathway to impact being deemed "unacceptable", requiring you to revise it before funding can be granted.

What are some of the most common mistakes people make in their pathway to impact and impact summary?

I've reviewed proposals for five out of the seven Research Councils and sat on funding panels for a number of Research Councils, EU and national governments. Here are a few of the most common mistakes I have seen:

No clear impact goals (or the goals are just about communicating the research to stakeholders or publics)
Benefits for researchers and the academy are included in the impact summary and/or pathway to impact, commonly including training and career benefits for early career researchers and students, and conferences and workshops that will mainly be attended by researchers. Cut and paste them into your academic beneficiaries section and start again. If you genuinely want to include capacity-building for your research team or students as part of your impact, explain how they will be able to use their skills and experience outside the academy to generate societal or economic benefits, and consider how you will achieve these benefits at scale, and evidence that they actually happen
Social science data collection methods are replicated from the case for support in the pathway to impact, claiming that the knowledge or engagement gained from these methods will generate impact

Public engagement for the sake of it—you have a clear pathway to impact via policy or industry and the reality is that your work is so niche, very few members of the public would be interested, but you’re going to bore the socks off a bunch of unsuspecting passers-by because you felt you had to add public engagement into your pathway to impact
Vague plans lacking detail are rarely credible
The impact summary is copied and pasted into the pathway for impact or vice versa
Even worse, copying and pasting from someone else’s pathway to impact

Finally, many people remove any impact goals and associated activities that are uncertain or high risk, leaving only a small number of highly conservative outcomes and activities, which fail to inspire or excite reviewers or panel members. Your funder will not expect to see every goal achieved in the same way as your research objectives, so the risks of dreaming big are relatively low, and the higher you aim, the higher you are likely to reach. You should, however, only ever promise to do things that are credible and feasible, and that you intend to actually pursue.