The impossible task of measuring impact

When making a donation to charity, most companies want their gifts to be used effectively and efficiently. Donations shouldn’t be wasted on unimportant things, but should instead be used to maximize the social impact of a particular cause. There needs to be a way to measure impact if you’re going to maximize it, but measuring impact turns out to be a particularly difficult problem. How should a firm measure the impact of their corporate philanthropy?

Impact is not cash

In spite of the data found in most corporate social responsibility reports, impact is not related to the size of the donation. One big U.S. corporation advertises, “over the last ten years we’ve given $175 million to charity,” which just generates more questions. And even if we can get answers to those questions we won’t really be satisfied. Let’s try anyway.

Q: Is $175 million a lot relative to the size of your company?

A: No.

Q: Why that amount?

A: It was what got approved in the budget.

Q: What did you spend it on?

A: 850 different programs in 40 communities.

Q: How did you pick those programs?

A: We have strategic giving priorities, and our grantees are selected after rigorous review of their grant applications.

“Hang on,” I hear you say, “I’m sure they get grant reports that contain some measurements.” They probably do, yes. But would that help us measure impact?

Q: What did nonprofit grantees spend the money on?

A: All 850 programs? A whole bunch of stuff. Arts, education, building houses, feeding people.

Q: Ok, so let’s just pick one ... feeding people.

A: Over the last 10 years hunger relief organizations have reported delivering 210 million meals with our funding.

Q: Is that a lot relative to the size of the problem?

A: No. There are about 800 million people around the world in need of food every day, so we solved seven one-thousandths of a percent of the problem.

That wouldn’t make a very good ad: “We’ve given 25% of the hungry people in the world one meal every ten years.”

Impact is not transformative change

Critics of hunger relief agencies in particular tend to quote the teach-a-man-to-fish proverb, and it’s easy to see why. If I give you a meal, you just get hungry again tomorrow. The real driver of hunger is poverty (or inequities in food markets, or the expense of transportation, or food waste, or maybe something else). Why not solve one of those problems instead? If you can achieve transformative change at the root cause then perhaps you can show real impact.

Except that you don’t. No matter which root cause you choose, you end up uncovering deeper and deeper causes. There are charities in each of these spaces and they’re addressing these causes with charitable transactions. For example, poverty can be addressed by providing jobs, so the new metric becomes jobs created. Those jobs might pay wages that are too low, so a different charity addresses that with living wage legislation passed. That causes employers to shift to more part-time workers, and you realize that you’re on the never-ending quest for the ultimate root cause.

Impact may not be subject to measurement

For some social issues, measuring impact might even be too expensive to be practical. One popular hunger relief program provides weekend meals for kids who receive free lunches at school during the week. It makes intuitive sense that kids who don’t eat during the weekend will do poorly in school. But there’s really no cost effective way to determine whether or not the kids are actually eating the food that they receive. In fact, evidence suggests that the food is being shared by multiple hungry siblings and sometimes even pets. So although the typical measurement is “bags of food sent home with kids,” the impact measurement is stymied because the kids may still be coming to school hungry on Monday. 

If we can’t reliably measure impact, how can we develop an effective giving strategy or know if we’re even helping?

Why we give

Even though we seem to have difficulty measuring the social impact of our charitable efforts, we continue to give. Researchers who study the psychology of donors think they know why. James Andreoni of UC San Diego has found that humans are rewarded with a pleasant feeling, or warm glow, when they help others. You can test this theory yourself. The next time you’re at the Starbucks drive-thru, pay for the car behind you, too. Even though you provided no meaningful social impact whatsoever and expect nothing in return, you’ll feel pretty good all day.  

In other words, identifying and measuring the impact of charitable activities might not be the most important part.   

Stories as impact metrics

Rather than trying to optimize the social impact of philanthropy, perhaps we should try to maximize the warm glow instead. The drive-thru experiment makes you feel good because there was an actual human in that car, who was shocked and surprised by the generosity of a complete stranger. That human will share a laugh with the Starbucks employee, will tell the story to a handful of people throughout the day, and may smile to themselves the next time they go through a drive-thru and are reminded of it.

In addition to generating units of social impact, each charitable transaction should provide a story that results in a warm glow. And figuring out how to get the best feeling from your philanthropy will ensure that you’re making a meaningful social impact.

As an example, let’s view the weekend food for kids program through this lens. After thinking about the needs of these kids and the limitations of the program, it still makes you feel really good to provide bags of food to hungry schoolchildren. Imagine what it’s like to be in second grade with no food in the refrigerator, but knowing that the snacks in your backpack will be shared with your little sister. That should make you feel good, and maximizing that feeling is a simple and effective way to measure your impact.

Both donors and nonprofits should continue to research and evaluate the actual social impact of charitable programs. But in the absence of hard data (or when collecting it is too expensive) it’s okay to rely on impact stories and their effects on you. Giving to a charity that makes you feel good will encourage you to keep doing it. If we can spread those good feelings, then we will all begin to make real, measurable impact.