Articles Learning what works and why from evidence of what doesn’t work

Learning what works and why from evidence of what doesn’t work

 

Howard White (the Campbell Collaboration) explains why context is everything when it comes to interpreting apparently contradictory research findings.

The Nurse Family Partnership – home visits to teenage first time mothers from low-income families – is presented in the book Moneyball for Government as an example of taking to scale a programme shown to work using rigorous evaluations. Such programmes are now common across the United States.

But a new study of the same programme in the UK finds that it doesn’t work. What are we to make of this apparent contradiction?

There is no contradiction. Context matters. And the relevant context here is ‘what is the counterfactual’? That is, what would happen to these low-income teens in the absence of the programme? The study designs are that the control group gets ‘treatment as usual’.  So in the UK the evidence shows that the Nurse Family Partnership offers no additional benefit to that provided by the free national health service, treatment with its free antenatal care, case free childbirth (with a bundle of goodies to take home), and free regular visits from a health visitor until no longer deemed necessary.  These things are not free in the United States, and low-income mothers may well not be covered by health insurance. So what we are learning is that providing ante-natal and post-natal support to mothers who would probably not otherwise get these services makes a difference. But if they do have access to services, then the additional home visits of the Nurse Family Partnership don’t have any effect.

There are no mixed findings, only incorrectly interpreted evidence. The general message here is that it is always important to be clear what the intervention is, and what the counterfactual, is in order to correctly interpret the evidence.

This evidence of course doesn’t mean that there shouldn’t be programmes for low-income mothers. We know that children from deprived backgrounds have worse life chances. And the early months and years matter a great deal for a child’s development. It is precisely because of this need that the nearly £2,000 per mother currently spent on the Family Nurse Partnership needs to be spent on an intervention which will make a difference.

 

Views expressed are the author’s own and do not necessarily represent those of the Alliance for Useful Evidence. Join us (it’s free and open to all) and find out more about the how we champion the use of evidence in social policy and practice.