What pumpkins tell us about impact measurement

The upside of not being able to sleep some nights, is that I get to listen to random features on the wonderful BBC World Service. Around Halloween there was an interesting programme all about pumpkins. And mainly about the huge amounts of waste involved when there is such high demand for pumpkins over a very short period. Did you know the majority of pumpkins bought at Halloween are carved up and then go to waste? Around 5 million pumpkins end up in landfill each year in the UK.

Pumpkin carving used to be the by-product of the edible pumpkin season but now the market has flipped on its head with decoration driving demand. The result is that pumpkins grown to be carved no longer taste nice. As one chef puts it: “Decorative pumpkins are grown with colour, structural strength, a flat bottom, and a sturdy stem as their main attributes. The flesh tends to be bland, watery, and fibrous. No one cares because they're going to be carved, not eaten.”

So what on earth has all this to do with impact measurement?

Measuring your outcomes and impact started off as an internal tool. It is a useful way for charities to decide where to allocate resources; where services can be improved; how your model benchmarks against others. It can also motivate staff and volunteers as they can see the difference they make. The by-product is that it is a good way to demonstrate to funders and other supporters the impact of your work. Funders often ask about how a charity measures its outcomes and how it uses this data to determine if this is an organisation that is still meeting a need, learning and adapting.

But just as with pumpkin carving, the situation is in danger of being flipped on its head with the wish to demonstrate the outcomes and impact externally now driving the process. Some charities are monitoring their outcomes not because they want to learn from this but because they know it is what they are supposed to do to get certain funding. I know I have been asked “what do you want us to measure?” And some commissioners are directing which frameworks or outcomes a charity must use.

If the external driver for impact measurement dominates, then we could end up with considerable waste as charities spend time gathering data that they don’t use, or implement systems in a token way. As funders, we need to convey the message that, whilst being able to understand the difference a charity makes is important to us, it is even more important that the charity adopts impact measurement systems that are appropriate for their clients and the nature of their work; proportionate to their size and resources; and that helps inform and improve their service delivery. We want our tasty soup first and then the lantern.