eyes on NYC

New York City’s community schools guru on the program’s massive expansion and why the schools are ‘here to stay’

PHOTO: Stephanie Snyder
Chancellor Carmen Fariña (middle) and Chris Caruso (right) visited East Flatbush Community Research School in 2016.

Chris Caruso is running one of the biggest education experiments in New York City.

The executive director of New York City’s community schools program, Caruso is responsible for delivering on one of Mayor Bill de Blasio’s core education promises: rapidly transforming hundreds of schools into community hubs with extra social services, additional learning time — and even washing machines.

It’s a model that has quickly gained steam. By September, 215 schools serving just over 100,000 students will be part of the city’s community schools initiative, which also encompasses de Blasio’s Renewal turnaround program. New York’s community schools program is considered a key national test case of whether the approach will pay off.

The responsibility to make sure it does rests partly with Caruso, who began working as a program director in a community school in Washington Heights nearly two decades ago. Chalkbeat caught up with him recently to talk about how he measures community schools’ success, what the program’s future looks like, and the challenges of quickly scaling up social services for thousands of students.

This transcript has been edited for length and clarity.

Chalkbeat: This is the largest community schools program of its kind. Do you feel a lot of pressure to make sure it is seen by the public as a success?

Caruso: I think anytime you work on public policy change in New York City, eyes are on you. So we’ve done pre-K at a scale that no one else has done, we’ve done after-school on a scale that no one else has done, and that’s part of the territory.

So yes, you feel that. But for me, and my team, and this administration, I think it brings us energy and drives us to show the potential of the success.

The program has obviously scaled up really quickly. Do you feel like there are tradeoffs in expanding that fast?

I think doing something at a large scale really positions you to have a number of people going through a similar experience at the same time, and we’ve seen real value in schools and partners in learning from one another.

If you were to pilot something in a handful of schools, you have the ability to really be involved and direct day-to-day operations, but you lose the ability to create this learning community, a diverse learning community, and so that’s something that we’ve really seen gain traction.

So now that this program has been off the ground for three years, what are the big problems you’re trying to address as the program moves into its next phase?

I think [one] of the things that we’re looking at for next year is how do we ensure a high level of service and quality across all the schools?

We’ve had principal transitions, we’ve had some [community-based organization] transitions, we’ve had community school directors change, and so we have a diverse pool in terms of where people are at in the stages of development of being a community school.

That’s something that as a system we need to be able to adapt and meet the needs of those schools. A specific thing that we’re doing around that is in our first two years of operation, we held a monthly conference where all the community school directors would come together and we’d do group learning, we’d do individual things, we’d have seminars.

Next year, we’re going to be changing that model and doing more cohorts based on where those schools are at, based more on geography. Getting 215 schools together is a lot harder than 150, but this will allow us to really differentiate our support in a more meaningful way.

You’ve said before that community schools shouldn’t be thought of as a turnaround strategy — something [former U.S. Education Secretary] John King agrees with. Does that mean that providing these extra social services, partnerships and programs is worthwhile, regardless of whether it produces academic gains?

This is an equity strategy. There are neighborhoods in this city where kids have access to far fewer resources, whether those are healthcare resources, learning experiences, relationship resources. And so community schools are a strategy to level that playing field. There’s evidence to back that up. A long-term investment in [services] leads to higher rates of attendance, lower rates of chronic absenteeism, greater connectedness to school — and all those things lead to better academic performance.

It sounds like you’re saying that these supports help create the conditions necessary for a long-term academic boost. Does that mean you’re not paying a lot of attention to [whether] test scores go up this year or next year?

I think it’s impossible not to pay attention to that. That’s the reality and we have that data. We’re looking extremely closely at chronic absenteeism and average daily attendance. And you know we’ve seen a decrease in chronic absenteeism of almost 7.5 percent since the program started. Citywide, [the decrease in schools is] less than 2 percent, so we’ve been really pleased with that progress.

We’re looking at school culture and climate and so we’re looking at the number of suspensions and incidents and seeing decreases there. And we’re very much looking at graduation rates and how students are doing. We’re seeing positive movement there and we expect that we will continue and that will deepen as the school culture changes, as kids feel more connected to adults and to their peers, and as they can see better and they’re healthier and they’re ready to learn.

Many of those [measures] are getting better citywide so it’s hard to know to what extent that is caused by community schools versus some of these broader trends. How do you try to separate that out?

One way we do that is you look at the schools in our portfolio and these are schools that are disproportionately serving children living in poverty, serving English language learners and students with disabilities, and you kind of look at growth among a cohort compared to citywide growth. And that’s one way that you can measure the differences between a particular intervention and the general progress that a district or a system is making.

You’ve said before that strong instructional practices are a key element of community schools. What percentage of your time is spent on thinking about that part of what schools do?

Strong instruction is what schools need to be doing, and so we have an infrastructure in the Department of Education through our superintendents and our Division of Teaching and Learning to support that. That’s not part of my core responsibility.

My role in my team is to help schools integrate partnership resources, and many times schools are looking to partner to support instruction. So that might be: How do you take a momentary break in the middle of a literacy lesson to get kids to be able to focus again? That’s an instructional practice, but that’s not about how you help kids get phonemic awareness. There are elements of kind of being present on managing emotions, on the social emotional skills, that we spend more of our time thinking about. I’m not writing math or ELA curriculum.

I’m curious how much has this model permeated the city more broadly? There are a lot of schools that have partnerships with community-based organizations but aren’t in the city’s official program.

There’s a cohort of schools out there that were implementing this model [before the official program launched]. One of the things we’ve tried to do in our scaling is to bring more of those schools into the fold so that they have access to the same types of supports as the other schools.

The number of schools that might not consider themselves community schools but that are looking at partnerships and that are looking at the whole child in a different way — I think that’s grown exponentially.

And so when Chancellor [Carmen] Fariña visits schools now and is asking a principal — regardless of whether they’re a community school or not — tell me about your [community based] partner, tell me about your after-school program, how are they helping meet the needs of your students, and how are you aligning your supports? That’s huge.

Does the city see this as similar to pre-K, where once you do it, it just becomes part of the system — a feature of New York City public schools?

Yeah, I think so. This mayor ran on that and we have a deputy mayor and a chancellor who have championed that. This is something, again, the fact that it’s not a solely a top-down approach, this is something that communities have been organizing around and advocating for for a while.

I think the depth of the roots of support are deep, and I think that we as a department now are organized around this. It intuitively makes better sense on how we align resources and support schools. So yeah, I think community schools are here to stay.

closures ahead

As New York City prepares to close more struggling ‘Renewal’ schools, here’s what we know about ones they’ve shuttered before

PHOTO: Alex Zimmerman
Monroe Academy for Visual Arts and Design, a Renewal school, was closed last year.

In the coming days, struggling schools in the city’s “Renewal” improvement program will learn whether they get more time to mount a comeback — or will be shut down for good.

New York City education officials are expected to announce soon which of the low-performing schools will close at the end of this academic year. The decision will have enormous consequences for students and teachers who will have to find new schools — and will likely rekindle debate about the effectiveness of Mayor Bill de Blasio’s $582 million effort to turn around troubled schools by infusing them with social services and academic support.

As the Renewal program passed its third birthday in November — a date by which the mayor promised to decide which schools aren’t measuring up — officials have been tight-lipped about which schools are on the chopping block.

Chalkbeat analyzed the previous rounds of closures — nine schools out of the original 94 — to understand which schools might be targeted this time. Perhaps the clearest finding is that it’s difficult to predict which schools the city will shutter.

While all the closed schools had very low graduation rates and test scores, so do other Renewal schools that were spared. The analysis shows there are no strict rules about which schools are shut down and which are given more time to turn around.

That said, here are some takeaways from previous Renewal school closures:

Almost all the closed schools struggled to retain students.

Seven of the nine closed schools enrolled fewer students in the year they were shuttered than when they entered the Renewal program in 2014 — and six shed more than a fifth of their students.

Many of the schools had struggled to recruit and retain students even before the program started — once it did, the schools struggled to staunch the flow. The city considers such shrinkage an existential problem; officials have suggested that schools with fewer than 250 students can become unsustainable since school funding is based partly on enrollment.

Six of the closed schools enrolled fewer than 200 students their final year — including Brooklyn’s M.S. 584, which lost about 25 percent of its population since the Renewal program started, leaving it with just 78 students.

Meeting the city’s goals doesn’t guarantee survival.

The city assigned each Renewal school annual goals around attendance, graduation rates, test scores, and other measures. (The goals have been criticized as overly modest.)

In the past, officials have said “all options are on the table” — including closure — for schools that fail to meet their goals. But those the city has actually shuttered have been all over the map.

For instance, the city closed the Bronx’s Leadership Institute the year after it hit 71 percent of its goals — more than most schools in the program. At the other end of the spectrum, it shuttered Monroe Academy for Visual Arts and Design, which met just 14 percent of its goals.

In the past, education department officials have said they consider a range of factors when deciding which schools to close, including their academic performance, feedback from families, staff turnover, and previous improvement efforts.

“When making decisions about school closures we carefully assess each school based on multiple measures,” department spokesman Michael Aciman said in an email. “In every case where we’ve proposed a closure, we’ve prioritized family engagement and guaranteed that every student has higher-performing school options.”

However, the mix of factors means it isn’t clear which schools are most at risk of closure. The fact that some were shuttered after meeting most of their city-issued goals only adds to the mystery.

Small gains in graduation rates and test scores aren’t enough.

A handful of shuttered Renewal high schools had boosted their graduation rates while they were in the program, while some middle schools got more students to pass the state exams.

However, the gains were usually small and the majority of students were still struggling.

At the Essence School, the share of students who passed the reading tests more than doubled since it became a Renewal school. But even with that bump, still only 5 percent passed. Meanwhile, math proficiency barely ticked up to 3 percent.

At the high-school level, every shuttered Renewal school saw an uptick in graduation rates.

The increases ranged from 5 to 17 percentage points. However, because most of the schools enrolled were relatively small, they could boost their graduation rates by several points simply by helping a few additional students earn diplomas.

And the schools with the biggest gains — a 17 point jump at Foundations Academy and a 13 point spike at Foreign Language Academy of Global Studies — came at schools with serious enrollment challenges. In their final year, both served fewer than 100 students.

research report

Three years in, some signs of (slight) academic growth at struggling ‘Renewal’ schools

PHOTO: Patrick Wall
Mayor Bill de Blasio at Brooklyn Generation School — part of the Renewal program

When Mayor Bill de Blasio launched an aggressive and expensive campaign to turn around the city’s lowest performing schools, he made a big promise: Schools would see “fast and intense” improvements within three years.

Almost exactly three years later, and after flooding 78 schools with more than $386 million in new social services and academic support, there are signs that the Renewal program has generated gains in student learning. The evidence is based on two newly updated analyses of test score data — one from Marcus Winters, a fellow at the conservative-learning Manhattan Institute, and the other from Aaron Pallas, a professor at Teachers College.

But the researchers caution that those improvements are modest — when they exist at all — and don’t yet match the mayor’s lofty promises.

The results may have implications far beyond New York City, as a national and political test case of whether injecting struggling schools with resources is more effective than closing them.

The two researchers previously reviewed the first two years of test score data in elementary and middle schools in the Renewal program: Winters found a positive effect on test scores, while Pallas generally found little to no effect.

Now, as the program reaches its third birthday, the pair of researchers have updated their findings with new test score data from last school year, and largely reaffirmed their earlier conclusions.

“We’re not seeing large increases” in student achievement, Pallas said. “And the reality is it’s hard to get large increases in struggling schools.”

Some advocates have argued that it is too early to expect big shifts in test scores, and that infusing schools with extra social services like mental health counseling and vision screenings are valuable in themselves. But de Blasio’s promise of quick academic turnaround has invited questions about Renewal’s effectiveness and whether resources can be more effective in improving low-performing schools than shuttering them.

To assess the program’s academic effect, Pallas compared changes in Renewal school test scores to other schools that had similar test results and student demographics when the program started, but did not receive extra support.

The biggest gains Pallas found were concentrated at the elementary level.

Over the past three school years, 20 elementary schools in the Renewal program have made larger gains on average in math and reading than 23 similar schools that didn’t get extra resources. The proportion of elementary school students considered proficient in reading at Renewal schools increased from 7 percent in 2014 to 18 percent last year — an 11-point jump. Meanwhile, the comparison schools also saw gains, but only by seven percentage points, giving Renewal schools a four percentage point advantage.

At the middle school level, the results are less encouraging. The 45 Renewal middle schools did not collectively outperform a group of 50 similar schools outside the program in reading or math.

In math, for instance, Renewal school students improved from 5 percent proficient to 7 percent. However, the comparison schools outside the program improved by roughly the same margin — increasing proficiency from 6 to 9 percent (and still far below city average). In reading, Renewal middle schools showed slightly less growth than the comparison group.

City officials have argued that Pallas’ findings are misleading partly because Renewal schools and the comparison schools are not actually comparable. Renewal schools, they say, were designated based on a range of factors like school climate or teacher effectiveness, not just student demographics and test scores.

“The schools included in the study are neither similar nor comparable in quality and a comparison of the two dissimilar groups is unreliable at best,” Michael Aciman, an education department spokesman, said in a statement. Aciman added that Renewal schools have made larger gains in reading and math than similar schools across the state, and have made progress in reducing chronic absenteeism and improving instruction.

Pallas notes that there are some limitations to his approach, and acknowledges that he could not account for some differences between the two groups, such as the quality of a school’s principal. He also does not use student-level data, for instance, which would allow a more fine-grained analysis of whether the Renewal program is boosting student achievement. But Pallas, and other researchers who have previously reviewed his data, have said his model is rigorous.

The Manhattan Institute’s Winters found more positive trends than Pallas, consistent with his earlier findings. Using an approach that evaluates whether Renewal schools are outperforming historical trends compared with schools outside the program, Winters found that the Renewal program appeared to have a statistically significant effect on both reading and math scores — roughly equivalent to the difference in student achievement between charter schools and traditional district schools in New York City.

Asked about how to interpret the fact that his results tended to be more positive, Winters said either interpretation is plausible.

“It’s hard to tell which of these is exactly right,” he said. But “neither of us are finding results that are consistent with what we would expect if the program is having a large positive effect.”