Evaluation in the Moment

Screen Shot 2019-07-10 at 1.17.46 PM.png

Freeing yourself from the post-event survey

Freeing yourself from the post-event survey

By Eleanor Hill and Laura Sundstrom

Screen Shot 2019-07-10 at 1.17.06 PM.png

You host a workshop or event. Because you are committed to continuously improving, you send out a post-event survey. But only a handful of your attendees complete it and those who do don’t provide meaningful responses. You’re not able to get useful or actionable information about what’s working and what’s not. So what can you do?

It’s time to try a new approach. In-the-moment evaluation helps you get feedback when attendees are most likely to engage – at the event. 

You can use different methods to execute an in-the-moment evaluation, but the key is designing opportunities that allow attendees to easily share their reflections, learnings, and suggestions in the moment. 

If you’ve ever been to an exhibit at a museum, you’ve probably seen an example of in-the-moment evaluation. The United States Holocaust Memorial Museum in Washington, D.C., has reflection cards (pictured) visitors can fill out to share what they will think differently about, what message will stay with them, and more. 

In this blog post, we share some guidance on when to take an in-the-moment approach, what methods to use, and the logistics you need to consider. 

Planning the approach

Before an event

Is it the right approach? An in-the-moment approach works well when you have questions about reactions to an event, experiences of an event, and intended behavioral changes because of an event. It also works well with an audience that will engage in a more creative and participatory evaluation approach. It’s important to design methods that are accessible, fun, and engaging for everyone. With careful planning, in-the-moment evaluation can be a great way to engage people and gather feedback from a diverse group. 

When and where? Depending on your questions, your method(s), the size, and the length of the event, it might make sense to conduct evaluation throughout the event, at a midpoint, or at the end. Think about when and where attendees will have time and energy to take part in the evaluation – asking people to participate as they head to lunch might not get you the best response rate! 

Who? How many people will you need and who will be best suited to conducting the methods that you are using? The skills for an in-the-moment approach can be different from the skills needed to conduct the same methods using a more traditional approach. You might be a fantastic interviewer but not the best person to approach and recruit people for mini-interviews at an event.

During an event

While facilitating your in-the-moment evaluation, make sure:

  • The methods are clearly highlighted and explained by the event organizers.

  • All speakers promote the evaluation to encourage participation. 

  • You have an evaluation team headquarters. This is especially important if it is a large event and the evaluation team is scattered about. You will want a place to check in with one another, store equipment, and download and record data. 

  • You take lots of photos! They can be useful when it comes to analysis and can make great visuals in a report. 

After an event

As with any evaluation, once you have collected the data you will need to analyze and report it. This will look different depending on the method you have chosen. For some methods, data analysis could be a small job. If you have used a polling app to get in-the-moment responses to a question, it could be as simple as writing the results into the report. For something like observations, the analysis will be a much larger job. 

Choosing the right method

Deciding which method to use for an in-the-moment evaluation can feel overwhelming. The best place to start is with what you want to learn. Here are some of our favorites.

You want to learn: How are attendees engaging at the event?

Try Structured Observations: Observe how attendees engage throughout the event or workshop, in a structured, intentional way. Create a guide of the types of information that would help you know if attendees were engaged (e.g., How many people pick each session? How many stay for the full session? Are people asking questions? Taking notes? Nodding? Doing the interactive exercises? Talking to the speaker afterward?). 

Resources Needed: You’ll need to train some volunteers or staff to observe these behaviors throughout the event. For large events, you’ll need more people in the observer role.

You want to learn: What are individuals learning?

Aisha BBH (4).jpg

Try a Learning Wall: Get written feedback from attendees on a large display. Take over a wall with a big attractive display, and ask people what the most exciting thing they are learning is. People can write their answers on sticky notes and post them on the wall or use dot stickers to agree with someone else. This not only captures feedback but can help spark discussion among attendees.

Resources Needed: During the event, you’ll need to staff the display with someone who is comfortable getting people to engage with the wall. You’ll also need time post-event to transfer the information to electronic format for analyzing.

You want to learn: What actions will attendees take?

Try Commitment Cards: Get personal reflections from attendees on cards. Hand out cards asking people what they will do differently as a result of the event or workshop: “As a result of your participation in ____, what one change will you make to your work?” Then ask attendees to turn in their cards at the end of the event. Ensure the commitment cards are easy for people to access. For example, add them to the badge holders (pictured).

Resources Needed: This requires less staff time during the event but lots of post-event time for typing up responses.

You want to learn: What about the food, speaker, and setting?

Screen Shot 2019-07-10 at 1.17.46 PM.png

Try a Suggestion Board: Get written feedback about standard event questions on a large display. First, think about whether you really need to ask these questions. Oftentimes, you already know the answer and have no control over the outcome. Was the chicken dry? Were the rooms cold? You’ll hear about it. But if you have to ask, try a suggestion board. Design a circle or square poster, and divide it into four parts labeled Keep, Add, Remove, and Change. Above the sections, write the question you’d like attendees to answer. Have people write their answers to your questions on sticky notes and up-vote others’ suggestions with dots.

Resources Needed: Just like with the Learning Wall, make sure you have someone staffing the board and getting people to participate, and transferring the information to electronic format for analyzing.


These are just a few methods for applying a in-the-moment evaluation approach. Feel free to get creative and come up with something that fits your needs, as long as you keep your end goal in mind: getting actionable data to improve your event.

Want even more ideas? Check out these awesome resources:


IMG_6402.JPG

Eleanor Hill, PhD | Evaluator

Eleanor is adept at seeing how social networks impact outcomes, applying network theory to understand big questions and gather helpful insights.

Laura_Site_480x320.jpg

Laura Sundstrom, MSW | Evaluator

Laura specializes in building evaluation capacity, helping clients understand the “why” behind evaluation tasks, and leaves an impressive trail of evaluation skills wherever she goes.

 

Laura Sundstrom