Section 1: Topic 1
The challenge of demonstrating REF impact for communications offices
While press offices might appear well positioned to contribute to the impact agenda, given their role in outreach, there are a number of challenges that limit the contribution press offices can make. To explore these challenges a survey administered by Fast Track Impact and The Conversation in December 2019[1]. Most survey respondents thought media engagement was very or extremely important for generating impact (80% of press officers, 70% of professional services impact staff and 100% researchers). However, there was widespread awareness of the challenges of media engagement as a pathway to REF-eligible impact among press office staff.
[1] There were 80 responses (31 press office, 26 professional services and 23 academics). While this is a small sample for the quantitative answers, the majority of the survey was open ended qualitative data, which was analysed thematically.
80%
of press officers
70%
of professional impact staff
100%
of researchers
The key reason given was the difficulty of demonstrating the significance of benefits for audiences, which may reflect their knowledge of REF rules around impact (most press office staff said that they knew a moderate amount (47%) or a lot (33%) about the role of impact in REF, with 7% saying they knew a great deal). The awareness of challenges was shared by professional services staff working on impact, but the majority of researchers responding to the survey seemed less aware of the challenges. Indeed, when asked to provide examples of significant impacts that could arise from media engagement, almost all the examples cited by researchers were pathways to impact (such as opportunities for further research or media engagement – see below), and only one respondent was able to suggest sources of evidence that could demonstrate the significance of media impacts.
47% knew a
moderate amount
33%
knew a lot
7% knew
a great deal
One researcher commented, “I'm not sure that they are invested in [impact]. Certainly in my institution, I think the media office is more focussed on getting branded stuff out to attract future students. There is a need in this context for a culture change to help make the media office more aware of why they might connect with researchers”. Other challenges related to the way the media operates, the focus of media engagement on reach rather than significance of impacts, poor targeting of audiences who could benefit or facilitate impacts, and challenges arising from press offices or researchers themselves (Table 1).
Crucially, the performance indicators typically used by press offices do not provide the kind of evidence needed to demonstrate impact for REF, which focuses on longer-term, wider benefits arising from media coverage. Circulation and viewing figures provide evidence of reach, but without evidence that there were significant benefits for those who engaged with the material, this reach has very limited value in the Research Excellence Framework. Impact claims based on reach alone are open to the critique that those who engaged may not have understood, acted on or benefited in other ways from what they learned, and so there may be no lasting impact from the work, no matter how impressive the coverage was at the time. As such, media coverage is typically seen only as engagement, or a “pathway” to impact, rather than as an impact in its own right.
Table 1: Challenges that may prevent media leading to impact
The way the media operates
Reach over significance
Poor targeting
Themes
Specific Challenge
Challenges arising from press offices
Challenges arising from researchers
Lack of relevance for REF due to issues generating or evidencing significant impacts (Table 1) was one of a number of reasons some 22% of professional services staff and 43% of researchers had at some point advised colleagues not to engage with media. Other reasons included concerns that: the researcher was too early in the research cycle to provide a robust message; the researcher may not have sufficient confidence or a strong grasp of the subject matter; and the research was controversial or sensitive and could have led to negative impacts or compromised future research (e.g. by losing access to the country).