There is a big debate building about how to treat evaluation in the social change space. It goes something like this: If nonprofits, foundations and social entrepreneurs want real change to happen, they need to get serious about whether their activities measurably contribute to the changes they espouse. They need rigor, RIGOR, in their evaluation methods! Once a critical mass of nonprofits gets serious about measuring its work, the marketplace of donors will leave those who lack rigorous reporting behind.

BUT, say others. But what are we really measuring? Can we reduce social change to indicators that are valued the same by everyone the way everyone agrees when a business reports its profits? In the words of one Skoll Forum speaker who I quoted week before last, isn't there something fundamentally different about philanthropy and social change? Unlike the marketplace, isn't a personal connection and a compelling story an imperative part of a successful transaction in the work we do?

So argues Shelagh Gastrow, who directs a nonprofit in South Africa.

I have no problem with measurement if it is done for the right reasons. These reasons will include that it comes from within the organisation, that it is done to assess the effectiveness of the organisation and that it is used as an organisational learning tool. The problem is, however, who is measuring, why are they measuring, what do they want to achieve by measuring and finally, can they effectively measure what they claim to want to measure?

Gastrow also takes on the idea of a marketplace for giving.

Why should grantmakers shop for the organisations that they want to partner with? It does offer a short cut, but as we know mail order brides are not always what they appear to be on paper. Unfortunately, there is no short cut. Getting to know organisations well enough to feel confident that funding will be well spent with levels of accountability is critical to the success of philanthropy.

This argument reminded me of a Skoll Forum delegate who participated in one of the consultancy clinics, pitching an idea for an online marketplace that philanthropists could use to discover new organizations in their areas of funding interest. The panel of consultants shot the idea down, citing similar attempts that failed in the past such as SeaChange (read the Kellogg Foundation's postmortem here) and noting that donors want to build relationships.

I think there is a middle ground in this debate. Organizations must not lose sight of what their end-goal is. More rigorous measurement that feeds back into an organization's efforts and helps it to stay on course toward its goals is indeed valuable. There may be commonalities between groups of organizations working on a particular social problem, but these measures will not be the same for everyone. Donors should get used to crafting measures specific to each grantee. With the exception of very specific goals, we'll probably never get to a place where everyone in the social change space values a set of common indicators enough to allow a meaningful social change marketplace. Powerful stories and strong relationships will continue to give meaning to the work we do. Just so long as there is substance enough to back them up.