Articles It’s time to standardise and regulate the Evidence Ecosystem

It’s time to standardise and regulate the Evidence Ecosystem

The contributions of Sir Joseph Whitworth to the industrial revolution are notable not because he built bridges, railways, or steamships but because the standardisation he brought to engineering enabled others to do so. Starting in the 1830s, the precision he brought to screw threads and machine measurement meant that the nuts and bolts of industry fitted together in the same reproducible way in Liverpool, New York, and Madrid, massively increasing efficiency, innovation and productivity. Without his advances, the pace and scale of change would have been severely limited.

Public service effectiveness and efficiency depend in large part on their foundations of the best evidence available, expert synthesis of this evidence, guidance derived from this synthesis, and the ability of services to respond to guidance quickly. Weak foundations in these areas mean that opportunities to improve services and minimise waste will be missed and that interventions which do more harm than good will be retained.

The effects of the COVID-19 pandemic on national economies are substantial. In the UK, Bank of England estimates are of a 14% shrinkage, and in the United States current forecasts, taking account of resumption of economic activity in the summer or autumn of 2020, are of declines in GDP of between 2.4% and 8.7% for 2020 relative to 2019. In this context, the efficient use of public resources is more critical than ever. During the pandemic, however, the importance of scientific evidence, the synthesis of this evidence, and guidance for government ministers and practitioners based on this has been established. This process has been subject to continuous public and professional scrutiny and has not just survived but is now seen as crucial to decision making by governments and by front line practitioners. It is now time to ensure that this process is robust in a public services context.

A new review of this process has concluded that robust, internationally recognised standards for evidence generation, synthesis and guidance production are available but have rarely been formally adopted. The only accreditation schemes are in UK healthcare and the UK government’s vibrant What Works Network of sector-specific What Works Centres and a central What Works Council. 

For policy makers, drawing evidence from many, often overlapping sources is currently a laborious, bespoke process; organisations which synthesise evidence have different evidence standards and vary in the extent to which they translate this into guidance. This unregulated evidence ecosystem is resulting in a deluge of evidence and guidance of variable quality and utility which can be ignored as a result.  

Recommendations flowing from this new review are:

  • Standards for evidence generation, evidence synthesis and guidance production should be set across the What Works Network and incorporated into the Network’s IMPACT principles. MHRA, CONSORT, Cochrane/Campbell, and AGREE standards should be the starting point in a systemic approach to this. 
  • Independent external review of the What Works Network should be commissioned and a comprehensive, streamlined quality assurance framework co-produced by the NAO or other suitable external organisation and the What Works Network. 
  • Organisations which carry out systematic reviews of evidence of intervention effectiveness and cost benefit should, by earning recognition through an accreditation process, be able to demonstrate that they meet Cochrane review standards. 
  • The NICE accreditation programme for guidance producers should be reopened to new applicants in the health and care sectors and adapted and extended across other What Works Centres and widely advertised in the sectors in which they operate. 
  • Technology appraisals, as carried out by NICE, should be used by other What Works Centres to assess technologies developed in their sectors, and the findings given statutory force, as with the findings of NICE technology appraisals in the NHS. 
  • The Research Assessment Framework in higher education should be adapted to facilitate assessment of the impact of What Works Centres. 
  • Methods used to increase safety in healthcare should be applied to ensure that distilled evidence and guidance derived from it prompts rapid change in the public sector. 
  • To increase responsiveness to authoritative guidance What Works Centres should develop formal relationships with service regulators and professional bodies in their sectors. 

Using a metaphor of which Joseph Whitworth might approve, the evidence ecosystem has also been described as an evidence grid, akin to the high-voltage electric power transmission network serving Great Britain which ensures that electricity generated anywhere can be used to satisfy demand anywhere. There is no doubt that evidence also has power to drive public service efficiency and effectiveness if it is generated, harnessed, and transformed to high standards.

Whitworth realised that standardisation and regulation would transform engineering. Public service transformation is just as achievable if the evidence ecosystem is subject to the same discipline.

Views expressed are the authors’ own and do not necessarily represent those of the Alliance for Useful Evidence. Remember you can join us (it’s free and open to all) and find out more about the how we champion the use of evidence in social policy and practice.