An Introduction to Measurement & Evaluation

If you’re a leader you know the feeling. You’ve been executing a strategy, implementing a program, or just performing business as usual and yet you have no idea how effective you are. You have an intuition you’re moving in the right direction but you really have no idea. For nonprofit leaders the feeling can be intensified since you don’t have a profit statement grounding you in reality.

Next week I’ll be doing a FREE 30 minutes webinar introduction to measurement & evaluation. M&E is all about looking, understanding, and maximizing impact. The impact of your programming. The impact of your strategy. The impact of your decisions.

The webinar will be a great introduction to some of the basic concepts in M&E and you will leave with some tools to help bring back to your team. We will be looking at:

  • Creating a Theory of Change
  • The role of M&E in strategic decision making
  • What data you should be focusing on

Space is extremely limited so if you are interested click here to register. The webinar will run from 12-12:30 CST on Thursday, March 8th.

Measure Outcomes, Not Activities

If you’ve read my blog for any amount of time, you know my love of data. I think that, while data should not have the last word on anything, it should have the first word on almost everything. It can bring clarity to an organization and challenge it to become better. Organizations around the world are jumping on the data train but sometimes they’re measuring the wrong thing.

The thing I see is organizations typically measure activities, not outcomes. That makes sense because activities are much easier to measure. Its easy to count how many people you served food too, how many people accessed your services, or how many volunteer hours you had this month compared to the previous month. This is good data to have but it does not answer the important question of the impact of your organization.

This activity oriented data doesn’t tell you how good the services that you provide are. They don’t tell you if people are better off after engaging in your services or not. You might be seeing increases in the number of people who access your services every year but that doesn’t mean that you are providing good services.

We need to address this question specifically in the social sector because it is fundamentally not a competitive market. The people paying for the services are not the ones consuming the services. So those that pay for and provide the services have to make sure that what they are offering is high quality.

If your organization is interested in engaging the question of impact, Contact Me.

Proof

Should non-profits be forced to prove that their interventions actually work? All too often the strategies and programs employed by non-profits seemed haphazard and have no real basis in scientific research. That does not mean however, that proof should be required.

Proof is a tricky thing. For one, attaining scientific proof of something is very difficult, requiring lots of time, money, and people. Second, as discussed in this post from Tactical Philanthropy, proof seemingly declines over time. “For all the perceived precision of a large study “proving” that something is true, the fact remains that over time our understanding of facts and truths change.”

The post quotes a New Yorker article title The Truth Wears Off that is quite interesting.

“The test of replicability, as it’s known, is the foundation of modern research. Replicability is how the community enforces itself. It’s a safeguard for the creep of subjectivity. Most of the time, scientists know what results they want, and that can influence the results they get. The premise of replicability is that the scientific community can correct for these flaws.

But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable. This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology.”

This is not to say however, that non-profits shouldn’t strive to understand the effect their interventions are having. All too often non-profits flee from facts and cling to anecdotes. They never step back and examine the impact they are having on the communities, people groups, or issues they are seeking to affect.

Scientific proof should not be the goal, but ignorance is unacceptable. A balance must be struck that seeks to understand affect without necessarily scientifically proving it.

Activities vs Outcomes

Activities are the things we do. Outcomes are the things we produce. Outcomes are more necessary than activities and today’s donors don’t want to just fund activities, they want to purchase outcomes.

An activity is handing out food at a food pantry. An outcome is helping people move from food dependence to food independence.

An activity is running an after-school basketball league. An outcome is increasing the odds a student ends up in college.

Activities make up the day to day life of social entrepreneurs and nonprofit leaders. Outcomes are why they started the organization to begin with. Read More…

Good Intentions or Good Thought?

Seth Godin has a great post today entitled Naive or Professional? I think he’s right on and it applies especially to the social sector. In the social sector though, I think we see the good intentioned and the good thought.

The good intentioned care, a lot. They sacrifice and they show up early and stay late. They give and give and give and yet always seem to be just scraping by. They seek to meet the immediate need. The good intentioned never have the time to stop and think about what they are doing because they are too busy caring for people.

The good thought measure and evaluate their work. They seek to improve themselves. They set boundaries and do what’s best for the long-term good of their clients and those they work with. They care but are able to detach enough to think clearly.

Good intentions are necessary but they are not sufficient when it comes to creating positive social change. Good intentions power people forward but good thought ensures that you are accomplishing what you care about.

Why Counting Matters

Not everything that can be counted counts and not everything that counts can be counted.

 

The above quote by Einstein is one of my favorites. I think its a nice counterbalance to the measure everything culture that some have dived into. What I love about the quote is that it is not saying that counting things is bad, just that there are some things that can be counted that don’t really matter and some things that matter that don’t really count. Read More…