In this guest blog, Rosie Chadwick draws on her own experience and shares her list of suggested barriers that need to be overcome before organisations can use evidence more effectively.
What gets in the way of organisations using evidence effectively? Based on fifteen years’ involvement in neighbourhood crime reduction, youth crime prevention and youth development programmes, here’s a suggested ‘top ten’ list of impediments that I think we need to overcome.
- Short-termism: too many initiatives that are here today and gone tomorrow, leaving insufficient time, money and motivation to put data collection systems in place, or – where they are in place – to capture trend data. In this context it’s worth reminding ourselves that the evidence base for Family Nurse Partnerships was more than two decades in the making;
- The fact that, even with longer-term programmes, outcomes may only be apparent long after an organisation has ceased working with a person or family, with whom they may no longer be in touch;
- The lack of interest funders sometimes show in ‘intermediate’ indicators – the small steps along the way that signal progress being made. The measurement system developed by Substance for the Home Office’s Positive Futures programme was an honourable exception to this, tracking young people’s movements along a scale from disengagement to autonomy. (In keeping with real life, there was also recognition that this movement could be ‘down’ as well as ‘up’);
- The vexed question of attribution, an issue that’s especially problematic for work in community settings which may be awash with interventions;
- Widespread confusion about what standards of evidence are good enough for what purpose;
- The lack of agreed ‘industry standard’ counting rules, hindering robust benchmarking of different organisations and approaches;
- In some cases, protectionism – hedging interventions about with licence fees and requirements that hinder their wider adoption. As somebody put it on the radio recently, where would we be today if Tim Berners-Lee had been proprietorial about his invention of the internet? Getting down to the rub’, I think that in the youth arena certainly we also need to acknowledge;
- A historically weak culture of performance management;
- A related lack of skills in data analysis; and
- A tendency to see performance reporting and review as divorced from service delivery – rather than fundamental to it. Asked to capture performance data, too often the reply that comes back is ‘I’m too busy delivering the service.’ Yet as a monitoring and evaluation specialist once put it to me very powerfully: ‘how can you possibly be providing a good service when you don’t know how effective you are being?’
Let’s hope that items 8-10 will feature in the recently announced review of skills in the voluntary sector, being led by Dame Mary Marsh, though I don’t think these deficits are the sole preserve of the voluntary sector.
Dare I say it, a silver lining of austerity may also be the stronger use of evidence, with a blunt message that ‘if you can’t prove it, we won’t pay for it’ allied increasingly to payment by results. I don’t think it’s any coincidence that one of, if not the best, example of Catch22’s use of evidence is our Payments By Results-based contract at Doncaster Prison. If we can frame Payments By Results targets in ways that benefit the most marginalised in our society then we will be well on the way to the effective use of evidence.
The views are the author’s own and do not necessarily represent those of the Alliance for Useful Evidence.