An education U-turn

Shut out but scraping by: Inside the struggling schools excluded from New York City’s biggest improvement efforts

Principal Taeko Onishi greets a student outside Lyons Community School. (Photo by Alex Zimmerman/Chalkbeat)

On a bright Tuesday morning, Principal Taeko Onishi propped open the heavy bronze doors at Lyons Community School, waving her students in with hugs and high-fives.

From this perch, Lyons looks like a thriving school. But on paper, it looks like one in trouble. Few of its middle school students are proficient in reading or math, as measured by state tests. At the high school, more than half of students were absent at least 10 percent of the school year. And when students graduate, 90 percent are not prepared for college.

“In many ways, we look like a deeply failing school,” says Onishi. But talk to her about the story behind those statistics, and it’s clear her school is serving some of the city’s highest-need students: those who are far behind their peers academically when they arrive, and are struggling with the effects of poverty.

Mayor Bill de Blasio has staked a huge portion of his education agenda on improving schools like Onishi’s. His administration has chosen roughly 215 schools to receive extra support through two overlapping programs that operate on a similar premise: Students can’t learn effectively if their out-of-school needs — from having clean clothes to basic medical services — aren’t being met.

Those two programs — Community Schools and School Renewal — both offer a massive infusion of resources for wraparound supports. But the city’s 86 current Renewal schools were chosen specifically because they had been among the city’s least successful schools for years. They receive extra academic help, stricter oversight, and are expected to show gains or face the possibility of closure. The city has spent $386 million on Renewal schools alone since the program was announced in 2014.

But Onishi’s school didn’t make the cut for either program. Instead, it is one of at least dozens of schools whose populations closely resemble the schools in the city’s programs in terms of need, but which have received starkly different treatment.

Getting into the programs is hardly a panacea. A Chalkbeat analysis found that schools outside these programs did not appear to fare worse academically, and their principals say they’re happy to escape the stigma of being labeled low-performing that often comes with the Renewal program, even as they envied the resources associated with it. The label can make it difficult to attract students, principals said, sending schools on a downward spiral that can end in closure.

Without Renewal or community school status, though, principals like Onishi are left to cobble together a range of social services largely on their own, with varying degrees of success. It’s an experience that highlights a downside of de Blasio’s resource-intensive approach: Some high-need schools are now better equipped to handle the challenges of poverty than others. “Our philosophy is [similar to] a community school,” Onishi added. “We just don’t have the money to do it.”

***

In announcing the Renewal program in 2014, de Blasio promised “fast and intense” improvements in nearly 100 struggling schools — identified based on their persistently low graduation rates, test scores, and the quality of their teaching and leadership. The plan? Rather than penalize them, he would flood them with the resources he said they desperately needed, closing them only as a “last resort.”

“If we do not see improvement after three years — and after all of these reforms and new resources — we will close any schools that don’t measure up,” de Blasio said at the time.

That was a major departure from the approach of his predecessor, Michael Bloomberg, who thought that only steep, immediate consequences could spur schools to change. Under his leadership, the city closed 157 schools it said weren’t working. Independent research later found that students benefited, but teachers and students were often demoralized — and the closures drew lawsuits from the city teachers union.

“It was really every school out for itself,” said Aaron Pallas, a professor at Teachers College. Autonomy in exchange for accountability, the theory went, was “the invisible hand that would result in positive performance.”

While de Blasio’s approach is less punitive — he’s only closed 10 schools thus far — it has been criticized for not producing changes quickly enough. Meanwhile, dozens of other schools have struggled with the same challenges as Renewal and community schools, without the same coordinated support.

PHOTO: Stephanie Snyder
The Richard R. Green campus in the Bronx

Take the North Bronx School of Empowerment. It’s one of three middle schools on the Richard R. Green campus in the Bronx. All serve an outsized share of students in poverty, who arrive on the first day of school far behind grade level, and struggle with high rates of chronic absenteeism and low state test scores.

But unlike the other two schools in the building, the School of Empowerment wasn’t chosen for the city’s Renewal program. And while the principals sometimes share teachers and after-school programs, there is a noticeable difference between the Renewal school and its non-Renewal neighbors.

The biggest one for that school’s principal, Magdalen Neyra: having enough counselors on hand so that each child has an adult in the building who knows their background. When a student doesn’t show up, there’s someone who can call home and coax them back to school — or intervene when the student is acting out.

“When [Leaders of Tomorrow] has that student in crisis, they have someone who’s a go-to,” Neyra said, referring to one of the Renewal schools in her building. “Even though they try to include us as much as possible, we haven’t been able to have that level of support.”

***

It’s not that Neyra and other principals like her are out in the cold. They’re benefitting from other education department initiatives, many of them new under de Blasio.

The city regularly sends a literacy coach to help revamp her school’s approach to reading instruction, which she credits with boosting state reading scores by 10 percent (though at 16 percent proficiency, the school is still far below the city and district averages). And, for the first time, Neyra’s seventh-graders were able to visit colleges outside of New York, through the city’s College Access for All program.

Brandon Cardet-Hernandez, principal of Urban Assembly Bronx Academy of Letters (which is not a Renewal or community school) said the city’s college access program has been a boon for his school as well. He’s used the extra funding to run SAT prep sessions, and hosts parent breakfasts at every grade level to walk families through various stages of the application process.

His school is also part of Single Shepherd, which has placed over 100 additional counselors and social workers in two of the city’s neediest school districts: District 7 in the South Bronx and District 23 in Brooklyn.

“I took it and did cartwheels,” Cardet-Hernandez said. The extra counselors have meant the school isn’t just responding to the most severe crises: a death in a student’s family, or a move into temporary housing. “We’re now talking about a student failing a class as a crisis.”

Deputy Chancellor Josh Wallack identifies these city programs, which are part of a constellation of initiatives known as “Equity and Excellence,” as the primary mechanism for improving schools that aren’t being flooded with new resources.

But critics say the school-by-school approach relies on programs that don’t touch every campus and, in some cases, won’t be fully phased in until well after de Blasio leaves office.

One initiative, for instance, aims to outfit every school in the system with computer science classes by 2025 — an example, one school leader said, of an initiative that doesn’t address core educational needs. “We barely have functioning internet,” the principal said. “There’s stuff like that that just doesn’t connect in a real way.”

Wallack pushed back on the idea that the initiatives are too fragmented and don’t represent as aggressive a strategy as the previous administration’s.

“It absolutely is a systems change with every bit of urgency,” he said. “Equity and Excellence for All is a chain of initiatives that will be there for children and families from birth through their entry into college,” he explained. “They all work together and we have a system for helping leaders turn them into a coherent strategy for each school.”

***

Even if they are helping, they aren’t always enough.

James Waslawski, principal of New Directions Secondary School in the Bronx, has used a state grant to bring in an organization that provides extra counselors for his middle school students, who are mostly over-age and off-track — a funding stream that will diminish next school year.

PHOTO: Patrick Wall
Principal James Waslawski designed New Directions Secondary School for middle school students who are over-age and off-track.

“We’re going to be scrounging this coming year to maintain the partnership,” Waslawski said, adding that he would likely scale back on bringing in outside coaches to help support his teachers.

School leaders like Waslawski may take it upon themselves to recruit social service providers. But even once those providers are in the building, schools are getting very different levels of support from the city to help manage the partnerships.

Principals in the city’s community school program (including Renewal schools) are given “community school directors” — extra leaders hired at each school who often take on key responsibilities for managing the school’s social services, identifying exactly what students need and lining up support. But schools outside the program — even those with social service partnerships — don’t automatically receive the extra staff member.

“I think there’s almost a requirement now that you’re identifying [community] partners,” said one Bronx principal who spoke on condition of anonymity. “You have principals who are focusing less time on instruction because they’re trying to manage and develop partnerships.”

That means they’re stacking one tall order on top of another, said Megan Hester, an organizer at the Annenberg Institute for School Reform who works closely with several community schools.

“Principals are instructional leaders and managers; they can’t all be expected to also have expertise in mental health, social-emotional development, community engagement, trauma, conflict resolution,” Hester said. “It’s already more than a full plate for principals of struggling schools to manage high-quality instruction for a diverse student population.”

Onishi, the principal at Lyons Community School (which is not, despite its name, in the city’s community schools program) knows that struggle firsthand. She said she works hard to offer her students active learning experiences, including regular visits to the Metropolitan Museum of Art that culminate in students giving their own parents a guided tour. But her staff often double as case managers, helping a student who got into a fight off-campus and faced assault charges, for instance, or one who didn’t have access to mental health medication after her mother lost health insurance.

Onishi says there are many other times when staff members don’t have the time to make the trip to court or to repeatedly follow up with a family after a student’s attendance slips.

“A lot of it just comes in from our staff who are willing to put in hundreds of extra hours,” Onishi said, noting the school’s counseling staff has grown, but is still not adequate. Three years ago, Onishi applied to be part of the city’s community schools program, which was recently expanded to 69 new schools, but her school was rejected.

“There will be other opportunities to begin to lay the groundwork to become a community school,” schools Chancellor Carmen Fariña wrote in a letter explaining the rejection. We “encourage you to keep your eyes and ears open for other opportunities as they arise.” But exactly how to break into the community schools program is unclear, said Onishi and other principals interviewed for this story.

When the city announced the latest round of community schools, Onishi says she was surprised by the news — no one had told her the program was expanding.

***

Meanwhile, few principals are clamoring to be part of Renewal, the city’s turnaround program that offers similar resources for low-performing schools. “I don’t think anyone wants that label on their school,” Neyra said.

Waslawski echoed that concern. “The things that they’re doing to encourage those Renewal school staffs to move in the right direction — that’s the right thing. But it’s all couched in the idea of failure.”

Being marked as a failing school can also hurt enrollment, some said, something many struggling schools already contend with.

“I wouldn’t want some sort of fancy name like a ‘support school,’ and you have to send out a mass mailing that your school is struggling — not in this climate with charter schools and school choice,” one principal said.

That’s a very real threat, say some people who work in Renewal schools. One Renewal school administrator, who spoke on condition of anonymity, said the program has brought much-needed counselors, social workers, and instructional coaches into the building — but has also made it harder to attract students.

“I suspect there is a lot of talk around the district,” the administrator said, referring to guidance counselors at feeder schools who might not recommend attending. Enrollment “is now going down low enough that it’s a major issue.”

Since funding is tied to enrollment, there can be significant consequences when parents vote with their feet. (Renewal schools have struggled to persuade parents to enroll; and to date 16 of the original 94 schools have been merged or closed.)

Other principals said they would rather retain their autonomy than receive an infusion of resources in exchange for the designation as a turnaround school, and face constant scrutiny or micromanagement.

“The message we have absorbed as principals, the message we’re getting, is you will lose the privilege to call the shots at your school that you see fit,” said one Brooklyn principal.

Some principals outside the program, however, said there were elements of Renewal they are envious of, including the chance to exchange strategies with other school leaders. “I’m not talking about visiting a school that is already high-achieving — I’m talking about a school that is just like mine,” said a different Brooklyn principal. “I don’t have that much opportunity to do that.”

And not all Renewal principals said the program has damaged their schools’ reputation. Some say it has led to improvements that could ultimately attract more families. Leaders of Tomorrow, a long-struggling Renewal middle school in the same building as Neyra’s North Bronx School of Empowerment, now has more than a dozen new mentors for students, a health clinic, and new professional development opportunities for teachers.

“We have hard data that we’ve looked at and the kids who are being mentored — their attendance has improved, their grade point averages have improved, their suspensions have dropped,” said Sean Licata, the school’s principal.

“We did not take the [Renewal] moniker as a bad thing,” he added. “It’s not the first thing we say … but with the services and the support, it’s an advantage.”

Neyra’s school could get the best of both worlds. Starting next year, Young Scholars Academy, one of the Renewal schools in the building, will be merged into her school. The move will come with some extra support but not the Renewal designation.

***

Extra services may help schools address the effects of poverty, but they don’t automatically produce academic payoffs. Pallas, the professor at Teachers College, found that the Renewal program did not yet appear to generate higher test scores or graduation rates compared with schools that have similar student demographics and were comparably low-performing when the program started.

“Based on the best evidence I see, the program is not having a meaningful impact on academic outcomes,” he said.

That doesn’t mean the program isn’t working at all, he added, noting that the social services may have value of their own, even if they don’t directly lead to academic improvements. Officials often point to other measures of success, including improvements in attendance and school climate, that could come before academic gains.

Just ask Onishi. She says adequate social services are a prerequisite for learning and without them, principals like her are swimming upstream.

“The biggest thing we need is support for families and kids for all these things that extend beyond the daily work of the school,” Onishi said. “We’re all kind of maxed out.”

approaching deadline

In year three of New York City’s massive school turnaround program, the big question is: What’s next?

PHOTO: Stephanie Snyder
Chancellor Carmen Fariña speaks with East Flatbush Community Research School Principal Daveida Daniel during a visit of the Renewal middle school.

On a recent Tuesday morning, Schools Chancellor Carmen Fariña walked into a classroom at Longwood Preparatory Academy in the Bronx, where students were absorbed with gleaming iMacs and recording equipment.

She paused for a moment, watching the teacher shuttle between students experimenting with audio-editing software.

“Look at the attention these kids are getting,” Fariña said, praising the school’s new vocational program in digital media. “It’s a feeling of renewed vigor and energy.”

With the smell of fresh paint still hanging in the air, Fariña’s visit was meant to highlight the enormous investments the city has made in dozens of schools that have floundered for years — including this one.

Under its former name, Banana Kelly, the school suffered from one of the highest dropout rates in the city, churned through four principals in five years, and struggled with serious safety incidents. (A previous principal was doused with pepper spray, and in 2012 was shot with a BB pellet outside the building.)

Now — as one of several back-to-school check-ins at some of the 78 schools the city is currently trying to revamp — Fariña was eager to praise the school’s energetic principal and its recent gains. Its attendance and graduation rates have improved in recent years, though its 2016 graduation rate (which is the latest figure publicly available) still lagged behind schools with similar student populations.

The visit comes as Mayor Bill de Blasio’s three-year effort to revive long-struggling schools, including the former Banana Kelly, is rapidly approaching its third birthday this November. The “Renewal” program — which has cost at least $383 million so far — is arguably the mayor’s most ambitious education reform, an effort to nurse some of the city’s lowest-performing schools back to health with extra social services and academic support rather than shut them down.

And while some experts say it’s too soon to expect big payoffs, de Blasio’s three-year timeline for “fast and intense improvement” has invited scrutiny into whether the program is translating into better outcomes for students.

PHOTO: Alex Zimmerman
Schools Chancellor Carmen Fariña tours Longwood Prep with Principal Asya Johnson (right) and student Heaven Molina.

“Enough time has elapsed that there is an appetite for looking at results,” said Aaron Pallas, a Teachers College professor who has studied the program.

Meanwhile, the exit strategy for schools in the Renewal program remains unclear. Despite promises that schools released from the program won’t lose the extra support they’ve come to rely upon, some school personnel are nervous that extra funding, counselors, and social services could be scaled back.

“There are some principals whose reaction is: ‘We really need to get it together because next year these [partner organization] resources might not be here,’” said Derek Anello, a program director with the nonprofit Partnership with Children who oversees staff in four Renewal schools. “Lots of people are experiencing it as the last of the three years.”

Planning for life after Renewal

Felicia Guerrier has helped usher in a new wave of social services at  P.S. 298 Dr. Betty Shabazz in Brownsville, Brooklyn, a Renewal school where 96 percent of students come from low-income families, and which used to suffer from “a love drought and a resource drought,” as she recently put it.

She supervises two new full-time social workers, oversees vision and dental services now offered out of the school’s auditorium, and coordinates with a nursing service that helps keep student health issues like asthma and diabetes in check — resources she says have led to a big boost in attendance and family involvement at the school.

But she’s already started handing off some of her responsibilities, out of a concern that the nonprofit she works for, Partnership with Children, might eventually be forced to reduce its role at the school. (Some nonprofits in Renewal schools are unsure whether their contracts will be extended past this school year.)

Guerrier explained that she’s now taking a back seat in meetings, making sure the assistant principal is in the loop to coordinate with health providers, and positioning the school’s parent coordinator to help run a food pantry for students and their families.

“I’m feeling the pressure to make sure there is some type of lasting power with what’s happening even when I’m gone,” she said.

While the mayor vowed in 2014 that the original cohort of 94 Renewal schools would be revamped or shuttered within three years (16 schools have already been closed or folded into other schools), the city has indicated the program will continue beyond this year. And officials stressed that the work of nonprofit organizations like Partnership with Children won’t end even if schools are taken out of the program.

“Steady improvement is key, and of course we will evaluate each school that is ready to transition from the program and provide them with the right supports to maintain their improvement,” Aimee Horowitz, the executive superintendent for Renewal schools, said in a statement.

An unclear exit strategy

Even as the Renewal schools move forward with their reforms, a big question hangs over them: How exactly do they exit the program? So far, no schools have left it without being combined with another school or closed.

This August, Mayor de Blasio said that would change. In addition to more closures, he said some schools could graduate out of the program, and others might stay past the three-year deadline.

“We have to work out the details,” de Blasio said, “but we’re not going to leave a school in the lurch.”

In making decisions about which schools to shutter or merge in the past, the city has looked at test scores, enrollment changes, principal effectiveness, and attendance rates — though officials have said there aren’t strict cutoffs, making it difficult to predict which schools could depart the program this fall.

If schools are released from Renewal, it remains to be seen whether they’ll continue to enjoy the same level of support and extra resources.

Brian Bradley, principal of Renaissance School of the Arts in East Harlem, said he is preparing for the possibility that this will be his school’s last year in the program. Renewal, he said, has made a real difference: Additional training for teachers has improved classroom instruction, aggressive outreach is boosting attendance, and the community school director has taken over once-overlooked administrative responsibilities.

“We have a great partnership and that has been the number one thing,” he said.

The school only banked on three years of support, but Bradley noted he has come to rely on extra funding the school receives to lengthen the school day. It’s a feature of the Renewal program that has a dual payoff, he said: more time for student learning and a pay bump that helps reduce teacher turnover.

While he’s already looking for sponsorships or other ways to fund his new programs, he’s aware that his school might have improved its way out of extra money and help.

“I have definitely used the phrase ‘victim of our own success,’” he said, “and that could be the reality.”

Looking for results

The program’s three-year anniversary doesn’t just create uncertainty for Renewal schools, it also raises questions about whether de Blasio’s signature turnaround program is working.

Some advocates of the mayor’s strategy have expressed concerns that his promise of rapid improvement was too aggressive. They say school turnarounds usually require well over three years, especially when they hinge on cultivating partnerships with social service organizations — a new task for many school leaders.

City officials have pointed to better attendance, test scores and graduation rates in some schools, but many others have not yet made significant academic gains. And researchers who have tried to sort out whether the program has led to academic improvements have reached mixed conclusions.

Using three years of test score data (including the results released last month), the Manhattan Institute’s Marcus Winters found that the program is generally boosting math and English scores in elementary and middle schools.

But using a different statistical method that compares Renewal schools to similar ones that didn’t receive extra resources, Teachers College’s Pallas found the program appeared to have no effect on test scores or graduation rates. (He has not yet updated his analysis to include the latest test scores.)

Renewal schools remain under pressure to raise their scores. To aid in that process, education staffers who work with the schools have pushed them to target increasingly specific groups of underperforming students, according to Partnership with Children’s Derek Anello.

“In the prior two years we were more general in what the goals were,” Anello said. Now, “There’s really a microscope on every number and how we move the needle.”

Yet even if schools don’t make huge strides this year, some observers say the mayor is unlikely to change course. Many argue that adding social services to high-need schools enhances students’ health and wellness, even if it doesn’t result in swift academic improvements. The city has invested heavily in creating social service-rich “community schools,” which include more than 130 schools outside the Renewal program.

Even if Renewal’s academic results are mixed, Professor Pallas of Teachers College predicts that de Blasio won’t face strong political pressure to scale back his resource-intensive approach to school improvement, which has generally earned support from local politicians and the teachers union.

The program “resonates with progressives’ desire to support community-based schools,” he said. But, he added, “at some point somebody’s got to make the difficult decision about whether the benefits are worth the investments they’re making.”

fact-check check

The NY Times tried to fact-check the mayor’s claims about Renewal test scores. Researchers say its analysis fell short.

PHOTO: Creative Commons / DMC Wilcox
The New York Times building

Reading the coverage of New York City’s Renewal program, it would be easy to conclude that the program isn’t working for most schools.

And on Thursday night, the New York Times continued in that vein, with a story about the turnaround program headlined: “For $582 Million Spent on Troubled Schools, Some Gains, More Disappointments.”

The story is framed as a fact-check on Mayor Bill de Blasio, who said at a press conference that Renewal schools are showing signs of progress since they had outpaced the city’s average growth in English and math scores.

In evaluating that claim, the Times points out that despite gains at some schools, most Renewal schools have actually not made progress closing the gap between their original scores three years ago and the city average. Some of the program’s fiercest critics seized on the analysis.

But according to three academics who study school performance, two of whom have studied the Renewal program’s impact, the Times’ characterization of the program as producing spotty results is problematic for the same reason de Blasio’s original claim of success doesn’t hold water. That’s because comparing Renewal test score data to city averages is poor evidence of whether the program is working.

The Times’ analysis can’t actually establish a causal effect.

The Times frames its analysis this way:

“To track the effects of the program, which gives schools a longer day and access to special services like vision care for students or mental health supports, The New York Times analyzed Renewal school performance on the 2016 and 2017 tests, as compared with the 2015 scores.”

The phrasing suggests that it’s reasonable to infer “the effects of the program” from test score changes, which is simply not possible, according to Thomas Dee, director of Stanford’s Center for Education Policy Analysis.

That’s because establishing a program’s effect depends on a model that can sort out what would happen to test scores without the program at all. One way to isolate that effect could be to compare low-performing schools that didn’t make it into the Renewal program with those that did, and study the difference in scores between the two groups of schools. But neither the education department nor the Times analysis attempted to do that, making those claims about the program’s impact misleading, experts said.

“There’s often this tacit assumption that we’ve learned about the true effect of the program from comparisons like this, and any researcher worth their salt will tell you that’s not the case,” Dee said.

Amy Virshup, a Times metro editor, defended the story’s analysis. “The piece never presumes to judge the success or failure of the Renewal program based on the ELA and math test results,” Virshup wrote in an email. “Judging whether Renewal is working or not would require many more data points and much more analysis.”

Renewal schools may be serving different students than when the program started.

Another reason the test scores could be misleading is that it has been well-reported that Renewal schools have lost a significant share of their students, continuing an enrollment drop-off that has persisted for years.

And since higher-performing students may be more likely to find a new school, it’s plausible that Renewal schools are serving a more challenging student body than when the program started.

“In general, the students that are most [likely to leave] are those who are higher performing,” said Marcus Winters, who wrote a report about the Renewal program for the conservative-leaning Manhattan Institute.

If that’s true, year-over-year test score comparisons wouldn’t be completely fair, since they would simply pick up changes in which students are served by Renewal schools — instead of the program’s real effect.

“The Times really didn’t do anything to ensure that their comparison to other schools was really comparable,” said Teachers College professor Aaron Pallas.

There’s no mention of rigorous research that has attempted to show causal effects.

Two researchers have tried to suss out whether the Renewal program is creating positive academic changes, and have reached different conclusions.

In an analysis that compares Renewal schools to similar ones that didn’t enter the program, Pallas found the program had essentially no effect on graduation rates or test scores. Meanwhile, using a different statistical model, the Manhattan Institute’s Winters found that the program is actually creating meaningful academic benefits.

The Times doesn’t cite either of those analyses, which would complicate the picture. The results of those research efforts suggest the Times’ description of mixed results is certainly plausible, but it isn’t directly supported by the data analyzed in the story.

Still, Winters said, the Times analysis is worth doing, as long as there are caveats, missing in this case, about what it can and can’t explain. “I don’t think it’s the definitive analysis of what’s going on,” Winters said. “But it’s not nothing.”