Shifting from Data-driven to Learning-driven

20181031_110035.jpg

The Evaluation Purpose Statement

The evaluation purpose statement

By Elena Harman

Evaluators cannot be both partners and judges. No one wants to share the truth of their success and failures with someone charged with grading their work—it is one or the other.
— Elena Harman // Vantage Evaluation

Evaluators cannot be both thought partners and judges. No one wants to share the truth of their successes and failures with someone charged with grading their work—it is one or the other. Even if you think the judge role is still necessary, we’ve learned from decades of trying to make accountability-based evaluation stick that it’s not what nonprofits need or want.

At Vantage, we see our roles as designing and executing the evaluation, then speaking for the data in conversations about what’s next. But it’s those conversations with program staff that generate the “judgment” and action plans, not us. We always set up the work to be learning-focused, not data-focused.

We have gotten to the point where evaluation is such a buzzword in nonprofit and foundation work that we assume all nonprofits should be evaluating everything, and for the same reasons. However, that is not true. Evaluation should serve an intentional purpose. It should teach you something new, and something that you care about.

I encourage the nonprofits I work with to create an evaluation purpose statement. An evaluation purpose statement is a two-sentence summary of why you want to do evaluation in the first place, and what the results will be used for. It’s a bit like a mission statement, but just for your evaluation.

The first sentence of your purpose statement should articulate what you want to learn through evaluation. Do you want to determine whether your program is effective? Effective at what, specifically? Do you want to know whether your program is being implemented appropriately? Do you want to determine whether it's reaching the right people? Solving a real problem? Making a longterm impact on the community? Changing the system? The clearer you can be at the beginning about why you're doing evaluation, the more likely you are to get what you want out of an evaluation in the end.

The second sentence of your evaluation purpose statement should include a clear articulation of how you're going to use the results. Are you going to actually use the results to improve your programs, or are they just going to sit on a shelf? And, if you don't have the power or position to improve your program, is it worth spending resources on evaluation in the first place? Are you going to use the results to communicate with funders? Donors? Beneficiaries?

Here’s an example:

Sentence 1: What do you want to learn through an evaluation?

Sentence 2: How will you use the results?

The purpose of the evaluation is to explore what areas of Positive Youth Development Program A impacts attending youth and which components of the program are the most effective.

Evaluation results will be used to advance programming to increase effectiveness and to report to funders.

The purpose of evaluation is to determine how effectively Program B builds capacity of local nonprofits to improve the quality of life for residents.

The results will be used to continually refine and improve Program B and, in turn, to effectively communicate what we do and why.

The purpose of the evaluation is to connect the dots between Program C processes and programs, short-term outcomes reported by families, and longterm outcomes highlighted in the research base.

The results will be used to articulate the strengths and weaknesses of Program C and, in turn, optimize our work.

The purpose of the evaluation is to detail what Nonprofit D does and which components of our work are the most effective.

Evaluation results will be used to advance programming by ensuring Nonprofit D can clearly define their activities, add value to clients and increase effectiveness by continuing to refine our programming.

The purpose of evaluation is to determine if and how Nonprofit E’s content inspires viewers to engage in civic action.

The results will be used to prioritize content and position Nonprofit E as a community leader and resource.

Additional Reading: Preskill, H., & Russ-Eft, D. (2015). Building evaluation capacity: Activities for teaching and training (Second Edition). Sage Publications.

It is critical that everyone within your nonprofit, particularly those involved in resource distribution (such as your executive directors and your board members), are bought into the evaluation purpose statement. The purpose statement sets the stage for why evaluation is important, and why it's worth investing financial and human capital. Purpose statements tend to convey a longer timeframe than key evaluation questions: A purpose statement captures the big-picture reasoning for five to ten years, whereas key evaluation questions might guide an evaluation for one to three years.

In the coming months, we’ll share activities you can do on your own to get started with effective evaluation planning. You’ll also find a comprehensive resource in “The Great Nonprofit Evaluation Reboot: A New Approach Every Staff Member Can Understand,” a new book coming in early 2019, from our CEO Elena Harman, PhD.


Elena.Site.jpg

Elena Harman, PhD | CEO

Elena takes the big-picture view of how Vantage’s work transforms how evaluation is used and perceived. She pushes everyone around her to think bigger about what evaluation can be, and how it can help improve our communities. With an encyclopedic knowledge of research and evaluation methods, Elena supports and advises the evaluation team on all projects. She connects the dots between data sources and projects. Elena has dedicated her life to Colorado and evaluation as a means to improve the lives of state residents. She brings a deep expertise of systems, nonprofits, and foundations in Colorado, as well as how to engage diverse audiences in a productive conversation about evaluation.


Elena Harman