In this guest blog, Naomi Eisenstadt discusses the use of evidence within the Government’s Sure Start programme.
The term ‘evidence’ is being loosely used in imprecise ways to argue for particular approaches to solving social problems: crime reduction, better social and emotional development, improved health outcomes. It is difficult to argue against using evidence to develop both policy and practice. But arguments will run and run about the nature and validity of evidence, what actually counts.
The last government was also interested in evidence and set up its Sure Start programme as an example of evidence-based policy. I have argued in my book[i] that while there was considerable evidence about the critical importance of early experience to children’s lifetime outcomes, the actual design of Sure Start was based on a range or other factors that were influencing policy making at the time. Labour in 1997 did not just want a raft of new policies, they wanted a new way of creating policy set out in the Modernizing Government Agenda. Reformed public services would be user not provider led, designed to deliver outcomes not inputs, work across institutional and professional boundaries, evidence based, and use cutting edged innovation. Particularly important for me was the greater involvement of outsiders, and bringing non civil servant experience into the heart of Government. This was demonstrated in the establishment of the Social Exclusion Unit and the Sure Start Unit, both of which included staff from the voluntary sector, local government, and a mix of civil servants from different departments.
The design of Sure Start made it particularly difficult to evaluate. Each local area was free to develop a range of services that would suit local needs. The engagement of local parents in the design and governance of programmes was meant to ensure that what was offered would be both appropriate to needs and acceptable to those who needed the service. It also meant programmes were very different from each other. Moreover, the original design was area based; Sure Start was meant for everyone within a catchment area with young children. The impact evaluation involved a random selection of families with young children living in the catchment areas using as a control group a matched set of children from the millennium cohort study. The random group of families selected would have included some families who never used Sure Start services. Sure Start was not a single manualised programme implemented with fidelity; it was a complex range of different services operating across 500 sites with huge variation in quality of implementation. Given these challenges, establishing any statistically significant evidence about the efficacy of Sure Start seems improbable at best. And yet there have been some positive results, particularly for parents. [ii]
There are several important lessons from Sure Start about evidence, policy and practice. A particularly relevant one to today’s debates is that some of the factors that make the biggest difference to child outcomes are actually the most difficult to change. Home learning environment has a much stronger impact on child outcomes than quality early education. However, despite the many evidence based interventions available, changes to home learning environments that in the longer term improve child outcomes are incredibly difficult to design and deliver. The latest evidence from the National Evaluation of Sure Start (NESS) is that home learning environments are improving for the original group of Sure Start families, but the expected improved outcomes for children are not as yet apparent.
A second key learning point which now seems obvious, is the importance of the quality of delivery, particularly the support and development offered to those leading local programmes. There is some evidence from NESS that well run programmes did deliver measurably better outcomes for children. But the speed of implementation meant that not enough attention was given to establishing the skill requirements to run such services. We were designing a new style of delivery of services to a large scale without initial or in-service training for those running the services.
Sure Start was established because the government of the time was convinced that the first three years of life can set up a child’s future for good or ill. The correlate argument about precisely what can be done to ensure that those first three years are as good as they possibly can be for all children is still highly contested. There is a huge amount of evidence NESS about what works for whom under what circumstances. As importantly, we also know a lot more now about what does not work. Yet the use of that evidence, as well as that from many other studies is patchy at best. Re-inventing wheels is actually fine. It is reinventing forms of transport that fail to get to the destination that is wasteful and depressing.
After spending several years working first in nurseries and then in management positions in children’s charities, in 1999 Naomi Eisenstadt became the first director of the Sure Start Unit. She has been Secretary of State’s Chief Adviser on Children’s Services and Director of the Social Exclusion Task Force. She has recently been advising the Big Lottery on their £165 million ‘A Better Start’ investment, and is currently a senior research fellow at Oxford University Department of Education and a Trustee at The Young Foundation.
[i] Providing a Sure Start, How Government Discovered Early Childhood, Policy Press. 2011
[ii] See www.ness.bbk.ac.uk for all Sure Start evaluation reports
The views are the author’s own and do not necessarily represent those of the Alliance for Useful Evidence.