How to get policymakers to use more evidence – a systematic review

 

A dearth of clear, relevant and reliable research evidence continue to block the use of research, according to a study of 145 research papers on evidence use. Difficulty finding and accessing this research, is also a major problem according to the authors of the review:  Kathryn Oliver, Simon Innvær, Theo Lorenc, Jenny Woodman, and James Thomas.

Despite several decades of work on evidence-based policy, the goals of improving research uptake and promoting greater use of research within policy making are still elusive.  Academic interest in the area grew out of Evidence-Based Medicine, dating back to Archie Cochrane’s book Effectiveness and Efficiency: Random Reflections on Health Services published in 1972. To describe the current body of literature in the area, we carried out a systematic review, updating an earlier study of the reasons why policy-makers use (or don’t use) evidence. We found 145 papers from countries all around the world, covering a wide range of policy areas, from health to criminal justice, to transport, to environmental conservation. We have concluded our findings below in the hopes of reducing the barriers of evidence-based policy.

What are the major barriers to policymakers using evidence?

A dearth of clear, relevant and reliable research evidence, and difficulty finding and accessing it, were the main barriers to the use of research. Access to research, managerial and organisational resources to support finding and using research were all important factors.

Institutions and formal organisations themselves have an important role to play. For example, decision-making bodies often have formal processes by which to consult on and take decisions. Participation of organisations such as NICE can help research to be part of a decision, but many study participants saw the influence of vested interest groups and lobbyists as an obstacle to the use of evidence. The production of guidelines by professional organisations was also noted in the review as a possible facilitator of evidence use by policy makers; however, in cases where the organisation was itself regarded as not being influential (such as the WHO) this was not the case. Whether true or not, these perceptions can damage an organisation’s ability to change practice or policy. On the other hand, institutions can make a positive difference by providing clear leadership to champion evidence-based policy.

Evidence comes from people we know, not journals

Good relationships between researchers and policymakers help the process of research use. This is a common finding; we know already from other studies that policymakers often prefer to get information and advice from friends and colleagues, rather than papers and journals.

In our everyday lives, we seek advice from friends and colleagues; we don’t trust people who have been wrong before, or who seem to be biased – or with whom we just don’t get along. Exactly the same applies to the process of policy making and, in our experience, in the process of research collaboration. Negative stereotypes abound on both sides, and our review found that personal experiences, judgments, and values were important factors in whether evidence was used.

Policymakers and researchers of course have different pressures, working environments, demands, and needs. The career paths and incentive structures are very different (and often don’t encourage them to work together, the ‘impact agenda

 Do definitions matter?

Policymakers tend to have a broader definition of evidence than that usually accepted by academics. Academic researchers, understandably, tend to think of ‘evidence’ as academic research findings, while policy-makers often use and value other types of evidence. For example, over a third of the studies mentioned that use of informal evidence such as local data or tacit knowledge.

Other factors reported to facilitate use of evidence are timely access to good quality and relevant research evidence. However, ‘making policy’ and ‘using evidence’ are both really difficult to do and to understand.These are not simple, linear processes – nor are they repeatable, predictable cycles of events

Figure 1,

[1] Figure 1: Linear stages of the policy process from the rational framework

Figure 2

[2] Figure 2: Examples of models of the policy process found in academia

The reality is likely to be far more complicated. The role of chance, e.g. who was available that day, or who you happened to have a conversation with, is hard to underestimate.

We need new institutions and rewards for evidence-use to bring about a sea-change in attitudes and behaviours on both sides, perhaps what is needed is a new way to think about creating and maintaining relationships.

This means new types of organisations, and new types of merit structures. In the USA, for an academic to leave a University and work at an organisation such as Brookings is seen as a promotion, not a failure. Australia’s Sax Institute provides a forum for policymakers and academics come together, similar to the ideas behind Cambridge’s Centre for Science and Policy. 

It remains to be seen how influential these organisations will be, and whether they succeed in changing the stereotypes and behaviours of both policymakers and academics. And critically, whether relationships brokered in this way can transcend the traditional ‘expert advice’ model of research mobilisation to encompass evidence that aims to be an accurate view of the totality of current knowledge on a problem – like this systematic review.

 

Kathryn Oliver
Kathryn is a Research Fellow in Sociology at the University of Manchester. She is interested in the debates about evidence-based policy, the role of policy networks, and ways of thinking about the impact of research on policy.

Simon Innvær 
Simon is an associate professor at Oslo University College, Norway. He is interested in evidence based policy, and the combination of process and effect evaluations.

Theo Lorenc
Theo is Provost Fellow in the Department of Science, Technology, Engineering and Public Policy (STEaPP) at UCL. His research focuses on interactions between knowledge communities.

James Thomas 
James is Assistant Director for Health & Wellbeing at the Institute of Education in London and Director of the EPPI-Centre’s Reviews Facility for the Department of Health, England, which undertakes systematic reviews across a range of policy areas to support the department. He specialises in developing methods for research synthesis, in particular for qualitative and mixed methods reviews and in using emerging information technologies such as text mining in research. Prof Thomas leads a module on synthesis and critical appraisal on the EPPI-Centre’s MSc in Research for Public Policy and Practice and development on the Centre’s in-house reviewing software, EPPI-Reviewer.

(The views are the author’s own and do not necessarily represent those of the Alliance for Useful Evidence)


 [1] 3 Linear Stages of Policy Process from the Rational Framework. Adapted from Jenkins (1978, p. 17)