Collaboration drives efficiency: Afterschool program providers pool data to measure impact across community

Collaboration drives efficiency: Afterschool program providers pool data to measure impact across community

There was a time when the Grand Rapids Creative Youth Center only had access to its own data.

The writing-based afterschool program evaluated itself using attendance numbers and pre- and post-program surveys, said Executive Director Kristin Brace. While the measurement was still helpful, the nonprofit’s analysis failed to tell the whole picture of its programs. 

That all changed when the organization was asked in 2014 to be a part of the pilot program for the Youth Community Data Center (YCDC), an online hub connecting data from out-of-school program providers across Grand Rapids.

With this project, afterschool or “out-of-school time” program providers have access to aggregate data for their own students from both Grand Rapids Public Schools and the Grand Rapids Police Department. The project also offers an average of the data across all after-school programs. Groups can use these numbers to evaluate the impact of their programming and then message that significance to funders and families in the community.

While a number of national players came together to create the YCDC, standing at the helm is the Grand Rapids-based Expanded Learning Opportunities Network, or ELO Network, a coalition of more than 60 organizations working to make out-of-school programs available to every child in the city. Some of the programs actually take place during lunchtime or other breaks, which is why they’re not always simply called afterschool programs.

The ELO Network, an offshoot of Grand Rapids-based Our Community’s Children, has worked for 15 years to build “a foundation of collaborative work and a very robust data agenda,” according to Executive Director Lynn Heemstra. This reputation drew the attention of the New York City-based Wallace Foundation, which then awarded a grant to the ELO Network several years ago. With this grant, the network turned to various universities for help on the aggregate data system that would come to be known as the YCDC.

The Community Research Institute at Grand Valley State University’s Dorothy A. Johnson Center for Philanthropy now handles the YCDC, both pulling the data and processing it. However, the John W. Gardner Center for Youth and Their Communities of Stanford University was also brought on for technical assistance to design the system.

Beginning with the pilot program, the Community Research Institute first asked ELO Network members what they would want out of a data system. Across the board, members needed something “low-burden and high-return,” as many afterschool program providers already are required to submit data to various places, said Amber Erickson, research manager at the Community Research Institute. They also wanted a way to relay the information to the public, she added.

The result is a system that collects data on school attendance, police contact, social and emotional growth, grades, and program attendance. Working closely with Grand Rapids Public Schools, the Community Research Institute is able to retrieve aggregate data for school attendance and grades for any given afterschool program provider’s students. The school system also has made available the average attendance and grades for all non-ELO Network students to better measure the impact of the afterschool programs. For instance, only 18.4 percent of ELO Network participants are chronically absent, compared to 31.4 percent of non-participating students.

Prior to the YCDC, afterschool program providers had access to none of this data.

“We’re receiving aggregate data that we wouldn’t otherwise be able to see,” said the Creative Youth Center’s Brace. “We can say, ‘Hey, our work really is having an impact. If you didn’t pick up our book and hear these amazing stories, here’s some numbers, too.’”

Before this project, the ELO Network had worked with the Community Research Institute on a report known as the Grand Rapids Juvenile Offense Index.

“National research was showing that the high periods of crime were happening after the school day, between the hours of 3 and 6 p.m,” Heemstra said. “We wanted to see if that was true for our city. We spent quite a bit of time with the (Grand Rapids Police Department) looking at police records, and in fact we did find out kids were getting in trouble after the school day and into the early evening. ”

To Heemstra, this proved the importance of afterschool programs specifically. This collaboration with the police department led to a partnership for the YCDC as well, in which members can see arrest rates for ELO Network students. That rate is currently 0.7 percent, with no comparison for non-ELO Network students yet available.

Another large piece of the YCDC is social-emotional impact. For this data, the ELO Network teamed up with Ron Ferguson of Harvard University with the support of the Believe 2 Become education initiative. Together, they developed a project called Imagine U, which helps survey participating students for their social-emotional status at the beginning and end of out-of-school programs. 

By every metric, ELO Network members have seen overall improvement compared to non-participating students. Before the YCDC, out-of-school program providers suspected this to be true, but didn’t have the data to back it up, Brace said.

The project has changed the way the ELO Network and its members function, according to Heemstra. They can evaluate what’s working and where there’s still room to improve. The capacity to do so will only grow as the YCDC gathers more data over the long term, she said.

“It’s pretty major because it’s system-building work. At the broad level, how do you align a lot of different organizations toward collective impact on children’s outcomes?” she said. “The ELO Network works because you have the commitment of community partners and organizations willing to come together, sit at the table and make this happen. … It really speaks to the power of community collaboration.” 


(adsbygoogle = window.adsbygoogle || []).push({});