So You Want to Be an Evaluator

AdobeStock_60406365.jpeg

FAQs from emerging evaluators

FAQs from emerging evaluators

By Elena Harman

As part of our mission to evolve how purpose-driven organizations use and think about evaluation, we meet and advise a lot of emerging evaluators. Many of these evaluators are eager to get started in the field and optimistic about the future potential of evaluation. And over the years, we’ve heard a consistent set of questions over and over again, so in hopes of supporting emerging evaluators near and far, we’ve consolidated our responses for you here.

  1. How did you get into evaluation?

  2. Do I need a PhD to be an evaluator?

  3. Do I need content area expertise?

  4. Where do I get evaluation-specific training?

  5. What skills should I focus on?

  6. Where do I find an evaluation job?

  7. What’s the best way to conduct an informational interview?

How did you get into evaluation?

I fell into evaluation immediately after college. In college, I thought I might become a research scientist… until I realized how ill-suited I was for year after year of bench work in hopes of making a difference in the world ten to twenty years down the road. So then I thought maybe I should be a doctor… but that wasn’t the right fit either.

In search of something I could love and make a career of, I headed to DC the summer before my senior year for an internship designed to bring more evidence-based policymaking to Washington. Policy work was a lot closer to something I could see myself doing, but I missed the data and the research, and I knew I was headed back home to Denver after graduation. So I started doing informational interviews to see what options might be available to me. Luckily, I had the chance to speak with the then VP of Philanthropy at the Colorado Health Foundation (whose children I babysat growing up). I explained to her that I wanted to do work that made a difference, that policy was interesting, but I worried I’d miss the research part of my life. And she responded, “what you are describing is a field, it’s called evaluation and it’s what you need to be doing.”

The rest is history: I was fortunate to be hired by the Foundation straight out of school, and my love affair with evaluation has continued ever since. When I was getting ready to leave the Foundation, I did informational interviews with every firm in Colorado that I had seen do evaluation work, and just couldn’t find the right next home. So I went back to graduate school to receive my PhD in Evaluation and Research Methods as a means to delay the “what’s my next job” question. While I was in school, I started to get requests for consulting support with folks I knew during my foundation days. Those projects evolved into Vantage, and the “what’s my next job” question answered itself.

To learn about the backgrounds of other Vantage team members, look in our blog for “Meet the Vantage Team” posts.

Do I need a PhD to be an evaluator?

No. You’ll notice that our evaluation team is about half PhDs and half Masters. At Vantage, we do not believe you need a PhD to be a good evaluator: what makes a good evaluator is a natural sense of curiosity, critical thinking, and the technical skills to carry out strong evaluation designs. We believe you gain those skills through a Masters in whatever field interests you, and some form of evaluation-specific training.

Note: you do need a PhD for some segments of the evaluation field, for example, if you want to be the Principal Investigator on a federally funded project.

Do I need content area expertise?

Vantage is a bit of an anomaly in our diversity of content areas. You’ll see our team works with clients across all content areas, and the content area expertise of our team is equally diverse, from anthropology to health psychology to network science to social work to art. It is more common for evaluators, and evaluation jobs, to have a more specific content area focus, so it can be helpful to start with a content area you are passionate about to help narrow your job search.

Where do I find evaluation-specific training?

  • Professional associations: Check out the American Evaluation Association (AEA), which hosts twice-a-year conferences and many virtual training opportunities. AEA also has local affiliates throughout the country, and that’s a great place to meet others in your region.

  • Certificate programs: A formal certificate program can be a great way to supplement your master's degree. The two programs I recommend are through Claremont Graduate University (a remote certificate of Advanced Study in Evaluation) and through the University of Denver’s Social Work School (the certificate has not yet launched, but it will soon and I’m really impressed with the thoughtfulness of the curriculum design).

    • For a more comprehensive listing of Evaluation certificates, masters programs, and doctoral programs, check out this guide from John LaVelle.

  • Stand-alone workshop programs: If a certificate is not the right fit, you can find stand-alone workshop options through Claremont (hosted in August each year, all webcast, and very affordable) and The Evaluators’ Institute (hosted throughout the year in various locations).

  • Internships: Find an evaluation internship to gain experience in practice. We host an annual internship program (returning in 2020), as does Public Profit, the Improve Group, and others (search AEA’s job board).

  • Books: If none of the above are the right fit, there are some great books to get you started with evaluation. My top three for emerging evaluators are mine (of course), Carol Weiss’ Evaluation, and Evaluation Roots.

What skills should I focus on?

The best evaluators are well-rounded, with both technical and interpersonal skills and diversity in both. Check out the AEA Competencies for a comprehensive list of skills to work on developing, or take the CDC’s Evaluator Self-Assessment to direct your learning.

Where do I find an evaluation job?

First, figure out what type of evaluator you want to be. Many emerging evaluators don’t yet realize how diverse the evaluation field is, and all the different places where evaluators practice. In a very oversimplified world, consider the following options:

  • Internal evaluators work for specific nonprofits and foundations. These jobs can be wonderful because they provide an opportunity to work deeply with one organization and see how evaluation impacts programming over time. On the other hand, most internal evaluators work solo or with small teams, so they can be isolating.  

  • Consultants (like us!) are one type of external evaluator that works independently from any specific organization but contracts with many of them. These jobs can be wonderful because of the diversity of work. Sometimes, it can be discouraging because once your contract ends you have very little control over the use of your work.

  • External evaluators within academic institutions. An example of this type of job is when an evaluator works for a specific University department and conducts evaluation as a part of that department’s research portfolio. These jobs can be terrific for evaluators with an interest in continuous opportunities for peer-reviewed publications, and with a passion for a specific content area. Challenges here are the same as consultant challenges, with the use of your work being out of your control once the project ends.

  • Government evaluators working directly for a governmental entity.  These often function as a bit of a cross between an internal and external evaluator: they may do evaluation projects for particular initiatives (like an external evaluator) and manage the strategic learning cycles for their department (like an internal evaluator). The challenge is generally the slow pace and bureaucracy of government work, which takes a specific personality to manage.

Once you’ve narrowed in, our whole team has found informational interviews to be a helpful route to navigate the field. Start with these common places to find evaluation opportunities, then consider reaching out directly to the organizations you are most interested in.

AEA job board and EvalJobs

Nonprofit Association Job Boards

Specific organizations’ websites

Internal

X

X

Consultants

X

X

Academic

X

Government

X

X

What’s the best way to conduct an informational interview?

First, do your homework! You should know the basics about the person with whom you are speaking before you sit down. Have a list of questions ready to ask them--use your evaluator curiosity to learn the most you can from each person. Take advantage of the time to gain a better understanding of what it is like working in the position/with the organization and how they landed a specific position. When I was first starting out, I did a lot of informational interviews by email, asking these five questions each time:

  1. How did you become involved in the evaluation field?

  2. What is one concept/theory/approach that has been highly influential to the way you think about evaluation practice?

  3. What has been one of your favorite evaluation projects and why?

  4. What advice would you give to someone new to the evaluation field?

  5. Can you recommend two additional evaluators for me to connect with?

Second, respect the person’s time. Be on time, and use the full time allocated to the interview. My biggest pet peeve is when I have an hour on the calendar with someone and they ask twelve minutes of questions and then say goodbye. On the other hand, take on the responsibility to end on time.

Third, send a thank you note and keep in touch. Not only is it one more opportunity to cement yourself in their head, but it’s also a great way to request any follow-up resources they may have mentioned during your conversation. I love when emerging evaluators add me on LinkedIn, so that I can see where they end up and how their career evolves.

Experienced Evaluators: What would you add? Reach us to us on Twitter to share your ideas!

Emerging Evaluators: Have more questions? Share them with us on Twitter, or reach out to us here.


Elena.Site.jpg

Elena Harman, PhD | CEO

Elena is the author of The Great Nonprofit Evaluation Reboot: A New Approach Every Staff Member Can Understand. She pushes everyone around her to think bigger about what evaluation can be, and how it can help improve our communities. Elena brings a deep expertise of systems, nonprofits, and foundations, as well as how to engage diverse audiences in a productive conversation about evaluation.