NEWS
EU seeks to evaluate the impact of its research
The European Commission is looking hard at how it can enhance and better evidence the impact of the research it funds.
A High Level Group on Maximizing the Impact of EU Research and Innovation is being set up to provide evidence that will inform the design of the successor to the €80 billion Horizon 2020 programme, which is due to commence in 2021. Following this announcement, Commissioner Carlos Moedas told the Conference on the European Research Area in October 2016 that the successor to Horizon 2020 had to be built on “a more sophisticated approach” to impact. He told delegates: “We have an obligation and an incentive to be much better at understanding and communicating the impact of what we do. Not only to ministers of finance, but to the general public.”
It is not clear to what extent reforms will relate to the application and review process, support for impact during the course of research, and/or evidencing of impacts during and after research has been conducted. The Horizon 2020 application process already asks how proposed research will generate impact and “European added value”. However, the current “dissemination” focus of many EU funded research projects could arguably, with support, move beyond dissemination to the generation of impacts. A more challenging proposition, given the scale of EU funding, would be any kind of comprehensive evaluation of impacts arising from this research. Commissioner Moedas appears to favour an approach based on metrics, which has been widely considered internationally.
Peter Strohschneider, president of the German Research Foundation, commented that introducing a REF-type exercise in Europe would be “very expensive” and lead to “scientific and intellectual shortcomings” if it were focused on metrics. “Questions of metrics are questions of power…This is all about political decisions on what we take as relevant in research”. However most countries that have investigated the use of impact metrics in detail have rejected this sort of approach as simplistic and unable to capture many impacts, particularly for certain disciplines. It seems likely that the EU will reach a similar conclusion in time.
Does the public agree with the value that researchers place on the impact of their research?
New research published in the British Medical Journal shows that the UK public value the impacts of research that are most prized by researchers.
The research asked medical researchers and members of the public to rank impacts, such as life expectancy and job creation, from best to worst. Using methods from economics, the team were able to show that the impacts most prized by researchers – improved life expectancy, job creation and reduced health costs – were also valued most highly by members of the public. However, there was less agreement between the groups on other impacts, including commercial capacity development, training and dissemination. The two groups were more likely to agree about impacts that were more far-reaching and significant in terms of their social benefit, rather than impacts occurring within the research system.
The research shows the potential for impacts that are judged to be far-reaching and significant (for example those graded as such in the Research Excellence Framework) to communicate the benefits of research to the public.
Pollitt, A., Potoglou, D., Patil, S., Burge, P., Guthrie, S., King, S., Wooding, S. and Grant, J., 2016. Understanding the relative valuation of research impact: a best–worst scaling experiment of the general public and biomedical and health researchers. BMJ open 6(8), p.e010916.
Might communicating your research to the public undermine your status as an expert?
Researchers from University of Muenster, Germany, have suggested that popular articles about research “make science too easy”, leading members of the public to “underrate their dependence on experts”.
The research, to be published in Public Understanding of Science, shows that members of the public are more likely to agree with knowledge claims after reading popularized articles compared to reading the original research. The authors say this demonstrates the “easiness effect of science popularization” which can lead members of the public to “rely too strongly on their own capabilities when making judgments about scientific claims”.
The article comes at a time when trust in experts is at an all-time low, with the rise of popularist, “post-truth” politics in many countries around the world. The findings present a dilemma for researchers who want to communicate their work and use their expertise for the public good. The research shows clearly that effective communication of research can empower publics to better understand and use research. However, the research asks whether there is such a thing as “too much empowerment”? This is likely to split a research community that is struggling to find a route back to relevance and evidence-based policy and practice.
Commenting on the finding, Clifton Bain, Director of the UK’s International Union for the Conservation of Nature Peatland Programme, who works regularly at the research-policy interface said, “Until now we focused on evidence; in a post-truth world we need to tell evidence-based stories, peer to peer”.
Rather than giving up on communicating research, the findings suggest that researchers need to change their approach to communication. Rather than pushing messages out through mass media channels and walking away, social media is enabling researchers to retain ownership of their own message, and engage in the conversation as their findings travel from peer to peer through social networks.
Scharrer L, Rupieper Y, Stadtler M, Bromme R (in press). When science becomes too easy: Science popularization inclines laypeople to underrate their dependence on experts. Public Understanding of Science doi: 10.1177/0963662516680311
Australian impact and engagement assessment pilot opens for business
The Australian government will pilot ways to measure the impact of university research and the universities' engagement with business and industry in 2017 ahead of a national rollout of the assessment system in 2018.
Announcing the pilot scheme, which will operate across ten broad disciplinary areas, Minister for Education and Training Simon Birmingham said, “The Engagement and Impact Assessment is about incentivising the smart and talented people working in our labs and universities to better focus on research that has wider economic and social benefits.”
A focus on “measuring” impact during the consultation phase has shifted to the “assessment” of impact, with a move towards the submission of case studies, despite concerns over the likely cost of the exercise.
Engagement and impact will be assessed separately. Assessment of engagement will involve metric indicators and a narrative statement. Impact assessment will involve qualitative case studies, supplemented with quantitative information where available. Submissions will be assessed by panels comprising academics and end-users of research.
A recent study of 45 Australian senior cancer researchers demonstrated mixed acceptance of their role and engagement with research impact activities, and highlighted potential problems of relying on researchers for collating and reporting impact data. Responding to the announcement of the pilot scheme, Chief Executive of Universities Australia Belinda Robinson said, “Some Australian research will result in direct commercial outcomes and some will not. That should never be the only test of the value of research."
New App enables “chance” collisions with people who can help you generate impacts
Like many researchers, James Eder believes in serendipitous impact. One morning on the Tube he sat next to someone who was preparing a CV and engaged the man in conversation. It turned out that he was exactly the sort of person Eder was looking for in his company, and a few weeks later they were working together. After this, the gregarious young entrepreneur started to wonder how many other people he might be sitting next to he could help, or who could help him.
Eder’s goal with new free app Causr, is to trigger impactful conversations with those around us (e.g. on train platforms, in airport lounges and cafes), leading to lasting working relationships and impacts that would not otherwise have been possible. He has given himself the title of Causr’s “Chief Collision Creator” because he believes people miss out on life-changing “collisions” with interesting people all the time.
The app uses LinkedIn to authenticate and populate your profile and lets you know when there are others with relevant interests nearby, who (by virtue of having the app) you can assume might be interested in meeting you. Causr is designed to act as an “ice-breaker”, giving you enough information about those around you (a job title, a shared university or a club membership) to give you the confidence to initiate a conversation. The app’s name is based on the Latin word for “motive”, causa. Eder says that “the idea behind Causr is cause and effect”. If we could identify a common cause with those around us, might we be able to work together to achieve the effects we want to see in the world? If the person you “bump into” has significantly more power than you to achieve those effects, and your research has something to offer, then perhaps we might be able to harness serendipity for impact, rather than waiting for chance happenings to occur.
