Building evaluation capacity leads to positive changes for the kids they serve
How the Denver Public Library’s Read Aloud program overcomes evaluation hurdles, leading to clarity and confidence in evolving their approach
Leaders at the library wanted more than a “gut feeling” that their program was making a difference. Find out how they overcame their evaluation paralysis, and what they are accomplishing now.
Introducing Denver Public Library’s Read Aloud Program
The Read Aloud Program provides research-based, early learning-enriched storytimes in approximately 350 early childhood classrooms across the city of Denver each week. This program aims to support early literacy skill development, helps kids learn to love to read, and builds social and emotional skills.
What they did
Make positive improvements to their program with confidence
After convincing their primary grant funder to include a one-time investment for evaluating their program, they experienced several false starts before seeing Elena, our CEO, present at a conference. They felt energized by Elena’s vision for evaluation, as well as her passion for helping nonprofits learn how to do evaluation well on their own.
After embracing a new definition of evaluation, learning new methods, and grasping how to analyze the data, the team was able to see clear trends. These findings gave them the confidence to make some significant changes to their volunteer training process and what they expect from their volunteers, all of which has lead to a bigger impact for the kids.
How we did it together
Overcome two major mindset hurdles that had been standing in their way
The first stumbling block was the internal departmental perception that evaluation clashed with one of their core values: the library is about creating an environment where everyone feels welcome to pursue learning in their own way, in their own time, and is not about testing kids. The second roadblock was the impression that evaluation has to be a complex, double-blind study showing direct impact on reading at grade level by third grade, and the fear that if they couldn’t do that, there was no point.
Together, we overcame these hurdles by:
Shifting the interpretation of evaluation from “testing” to “learning in our own way.”
The team came to evolve their view of evaluation from something that clashes with their core values to one that celebrates them.
Learning qualitative methods that felt authentic to the organization.
Training on focus groups showed them a way to embrace the stories that felt important, but that they had previously believed had no place in evaluation.
Tweak the tools to make their evaluation efforts more impactful
The team experienced many a-ha moments together, and we modified the evaluation tools they already had in place.
Ask more intentional questions to get more valuable insights. Rather than asking teachers about general satisfaction, they now drill deeper to input around early literacy and social-emotional skill development. This has inspired improvements to the program.
Tweak the tools to get a higher response. Small, but significant changes to their surveys have led to an increase in quality data.
For example, shortening surveys, adjusting how and when they send them, and closing the feedback loop afterward.
Build skills in analyzing the data. Learning to use pivot tables and make graphs in Excel allowed them to visualize the findings, as well as reach a deeper level of understanding.
Deeper understanding leads to considerable changes around the volunteer component of their program
The evaluation had concrete evidence, rather than just “gut feelings.” These findings gave the library team the confidence necessary to make significant changes to their volunteer training process, as well as what they expect from their volunteers. This change led to a bigger impact for the kids.
Shift program elements from optional to required. A profound understanding of the impact singing and dancing has on early literacy skill development inspired the team to make this a requirement, rather than an optional bonus activity.
See volunteers in a new light. As findings led to increasing program requirements, the team gained courage to expect more from volunteers, understanding they are a critical component of running a successful program.
Significant improvements to their volunteer training. From the content to the amount of training offered per year, and expectations for how frequently volunteers must attend these trainings, they have made substantial shifts for more impact.
New evaluation skills lead to ongoing greatness
The capacity building project has led to new skills and a positive shift within the team, which will continue to pay dividends for years to come.
Evaluation is part of ongoing discussions.
Now, they are thinking about evaluation at the beginning of a project, and have inspired other programs and departments to rethink how they evaluate.
More effective grant applications.
The team can now present specific data regarding the impact of their program, rather than having to rely solely on stories and pictures of cute kids.
Proactively applying new evaluation skills.
They have conducted their own focus groups, and felt empowered to apply the concepts to improve their site observations of volunteer readers on their own.