Articles The problems with impact measurement – how only the right data can help charities

The problems with impact measurement – how only the right data can help charities

 

Jenny North, Director of Policy and Strategy at Impetus Trust- the Private Equity Foundation, explains focusing on the right data and impact management can transform charitable organisations. 

Charities operate in multiple markets – and none of them could be described as sane. The general public donate largely based on name recognition and in ignorance of outcomes. Trusts and foundations are notorious for making funding decisions against their own, sometimes opaque, aims and objectives. Contracts and Service Level Agreements at local and national government level frequently contain perverse incentives, and can require the capture and counting of large datasets which serve no useful purpose for the delivery organisation – or really, for the commissioner. Add to this the fact that the problems most charities are trying to solve are complex, and multi-factorial, with little consensus on how best to address them. Staying on-mission, and delivering against that mission, is a constant struggle.

Impact Measurement
The rise of ‘impact measurement’ has led every funder to demand more data, usually to make the funder themselves feel effective. This haphazard, compliance-focused approach to performance management does charities – and the people they exist to help – no favours. Counting things which aren’t important to you, and which you don’t use to manage resources is a colossal waste of limited charity resources. Too often, this is what impact measurement looks like.

At Impetus-PEF we believe that the right data, and its use in performance management, is the key to creating high-performing organisations which consistently deliver socially significant outcomes for beneficiaries. But too few charities are collecting the right data – or often they haven’t even identified what is the right data. Further, they are not using the right data in their performance management practice, and this prevents them from becoming truly high-performing organisations.

So what data does a charity need to determine if it’s having a significant and positive effect?

In a nutshell, you need data which tells you who you’re serving, what you’re giving them, and what are the short, medium, and long-term outcomes. This data will not ‘prove’ you are having a significant positive impact– for this you need a summative evaluation, such as a Randomised Controlled Trial. But it will tell you whether you are delivering what you intended to whom you intended, and whether it seems to be producing the changes (for that’s all an outcome is) you intended.

Theory of change
The critical step which needs to come before even this, and one which is often overlooked, is the development of your organisation’s Theory of Change. This process will define what your organisation is intending to achieve (outcomes) for whom (target population) and how (your services). Defining this goal in measurable (not ‘visionary’!) terms is critical for being able to know what data to collect to monitor how you’re delivering against your mission.

Then, crucially, collecting this data in real-time allows you to manage your performance – ion adapt the service an individual receives, or to change your engagement practices based on feedback data. In this way you can ensure beneficiaries benefit as intended – not just check afterwards whether they have or not.

Impact management
This data, and impact management, can become the glue which holds together an organisation. It bonds the frontline workers who see beneficiaries every day to the managers who ensure the frontline has the resources (and the challenges) it needs, to the leaders relentlessly questioning themselves and their colleagues, as to whether they are doing the best they can for the people they are helping.

Staying sane in an insane world is a day to day job for non-profit leaders. The right data keeps your efforts from being diverted towards someone else’s agenda and outcomes. It keeps you and your staff honest about where your performance needs to improve. And, crucially, it keeps your eyes focused on the difference you went into this business to make, and shows you how to make more of a difference year on year. The right data can transform your organisation – and it can help you sleep at night.

*With acknowledgements to David E.K. Hunter for the title, and for his patient explanations of impact management.

 

Views expressed are the author’s own and do not necessarily represent those of the Alliance for Useful Evidence. Join us (it’s free and open to all) and find out more about the how we champion the use of evidence in social policy and practice.