Section 4: Topic 3
Identify how you will create evidence to demonstrate impact
Evaluation is built into the Fast Track Impact impact planning template. As you consider what impacts you want to see (point 1 above), and the kind of media engagement you’ll need to get those impacts (point 2 above), you need to consider how you’ll know if the media engagement is working, and whether or not it is generating the benefits you’re aiming for. Press offices have significant expertise in collecting data on the reach of media engagement, including circulation and viewing figures, and social media metrics. Increasingly there are a range of other ‘altmetrics’ that you can get around media engagement that will tell you more about the number and types of people who are talking about the work, and where they come from. These kinds of metrics can be a useful starting point for evaluating the significance of impacts arising from media engagement. However, in isolation all this data really tells you about is reach.
​
This is a key weakness of traditional media monitoring, because without evidence of any benefit for the millions of people that were reached, it is difficult to argue that there is any impact based on the technical definition of impact (e.g. from REF) or academic definitions of impact, which all emphasise the significance of benefits as well as their reach. Based on the points above, you have now created a plan that uses media engagement in a much more targeted way. Now, you just need to make sure you can measure the significance of those benefits in addition to their reach.
The logic model approach makes this easy, because it asks you to identify indicators of success, which do not require any specialist expertise. If your impact goal is to change people’s awareness of the benefits of maggot therapy and shift attitudes to reduce levels of disgust and increase the likelihood that a patient might accept the treatment on the NHS, then you know exactly what you are looking for as an indicator of impact. In this case, you would expect a sample of people exposed to your media coverage to have different levels of awareness and changed attitudes before and after engaging with the coverage, and you would then ultimately expect to see an increase in uptake of the therapy in hospitals. Developing indicators like this does not require any social science expertise – just common sense. Ask yourself what success would look like, and then ask how you might be able to evaluate that. In some cases there might be things you would be able to quantify, and in other cases you might expect people to say certain things qualitatively if your work had made a difference to them. What methods would you then need to collect that data?