This article was originally published in The Notebook. In August 2020, The Notebook became Chalkbeat Philadelphia.
Schools turned over to charter operators — and to a lesser extent, District-run Promise Academies — have shown improvements in academics and climate under the three-year-old Renaissance schools turnaround initiative, a new report has found, although big first- and second-year gains have started to slow down or reverse.
According to the study, conducted by the District’s Office of Research and Evaluation, most Renaissance charters continue to have higher proficiency rates than those schools did pre-turnaround, despite the leveling-off of earlier gains.
The reported improvements occurred during a time when overall proficiency rates for District-run schools were declining after years of increases; the downslide began after strict test protocols were put in place in District schools in the wake of a statewide cheating scandal.
As a group, the seven schools in the first cohort — those converted starting in 2010 — posted the most dramatic gains. The academic results for the most recent cohort studied — the four schools converted to Renaissance charters in 2012 — were decidedly mixed.
The report indicates that the Promise Academies model showed some modest success at the start, but has been plagued by "an unprecedented budget shortfall" and "lack of clarity about what distinguished Promise Academies from other schools."
Since the Renaissance initiative began in 2010, 17 elementary and middle schools and three high schools have been turned over to charter operators. In addition, eight elementary and middle schools and seven high schools were named as Promise Academies for intensive "in-house" turnaround. But three of the Promise Academies, all of them high schools, have since been shut down.
Three of the Renaissance turnarounds and six of the Promise Academies just started this year and were not included in the evaluation. There were nine Promise Academies and 17 Renaissance schools operating in 2012-13 and covered in the study — either in their first, second, or third year of turnaround.
The report found uneven performance among the seven Renaissance providers.
Mastery — the largest provider, with six of the schools in the study — showed across-the-board double-digit gains in all its schools in both math and reading, although those gains for the most part had begun to stall or reverse in the third year. Universal, which runs five schools, showed reading gains in four of the five but math declines in three. ASPIRA, with two schools, showed gains in both, although one school’s results began to slide in the third year.
Young Scholars’ school, Frederick Douglass, showed big math gains in the first two years and smaller reading gains. But last spring in year three, the math gains reversed precipitously and the reading gains flattened. Mosaica’s Birney Elementary, which started as a Renaissance school in 2011, in the second cohort, has shown steady gains in its first two years.
String Theory and American Paradigm had completed one full year running one school each. Math and reading proficiency rates at String Theory’s Performing Arts Charter at Edmunds were mostly flat. At Memphis Street Academy, formerly John Paul Jones Middle School, math proficiency declined substantially while reading scores were flat.
Of the three high schools turned over to charter operators in 2011, both Gratz, run by Mastery, and Olney, run by ASPIRA, showed gains in math and reading in the year after the turnaround (2013 PSSA scores were not available for the high schools).
Audenried, run by Universal, however, showed declines in both subjects. Audenried had started out with the highest proficiency rates of the three Renaissance high schools but slipped to the lowest.
Noting that the concept of turnaround relies on "dramatic" and sustained gains, the report’s authors expressed concerns about whether Universal and Young Scholars, based on the first three years’ performance, could maintain an upward trajectory at their schools. "Changes … were positive in the first two years, but those increases have not been sustained," the report said. "As such, these schools may fall short of what is needed to achieve dramatic results within 5-6 years."
It also describes the schools run by String Theory and American Paradigm in their first year of turnaround as "floundering."
The report said that all Mastery schools remain on an "upward trajectory" — despite third-year declines or flattening out at most of its schools.
Alfredo Calderon, executive director of ASPIRA, said that he had not yet read the report, but was not surprised that the early gains are leveling off.
"We expected that," he said. "Just by making the change we make immediately — safety first — kids feel, ‘I can come to school.’ They feel [they] can learn, they don’t have to worry they’ll get beat up, we’re bringing in teachers who want to teach — automatically the scores go up," he said. "There’s a full change of culture, discipline, educational philosophy."
Although the study relied heavily on PSSA scores, it did review other measures, including retention of students and climate indicators.
It found that, after turnaround, both Promise Academies and Renaissance charters were more successful in keeping students enrolled for the entire school year — so-called "within-year retention." The Renaissance charters had a greater increase. The Promise Academies improved within-year retention for 9th graders, but not for students in the elementary and middle grades.
Although the report concluded that, overall, there was not a big change in the demographic makeup of students attending Renaissance charters before and after turnaround, researchers found that there has been a statistically significant decrease in the percentage of English language learners — from 8.6 percent to 7.2 percent — and an increase in the percentage of special education students — from 16.8 percent to 19.9 percent.
At both the Renaissance charters and the Promise Academies, about 80 percent of the students who attended the schools pre-turnaround continued to attend after the conversion, the data show. Some of the Renaissance schools showed increases in enrollment, which suggests that more students from the area who hadn’t enrolled before decided to attend after they were converted to charters.
Most Renaissance schools, both charters and Promise Academies, also showed improvements in school climate, with almost across-the-board declines in reported serious incidents per 100 students and in the number of student offenders. The biggest declines were at Renaissance charters; there were no serious incidents at all in 2012-13 at Smedley Elementary, run by Mastery, and Edmunds, run by String Theory. At five of Mastery’s six Renaissance schools, the percentage of students who were "offenders" involved in serious incidents was 1 percent or less.
A big exception to the positive trend in serious incidents was Birney Elementary in Logan, run by Mosaica, which showed a fivefold increase in serious incidents, although Birney’s academic indicators showed significant gains.
The report noted that at the Promise Academies, resources were severely cut back in 2011 after the first year of the initiative.
Most of the Promise Academies showed gains in the first year that then leveled off or declined. At three of the first four Promise Academies — Ethel Allen, Clemente and Potter-Thomas — proficiency rates have declined to below pre-turnaround levels. Only at Dunbar were the first-year gains sustained.
"It is possible that the Promise Academies are suffering more from poor fidelity of implementation than outright failure of the turnaround effort," it says, noting that the model has been altered each year. "It is likely that the charters have benefited from a clarity-of-purpose that has been lacking at the Promise Academy schools."
ASPIRA’s Calderon also noted that that at his schools, 40 percent of the students are English language learners (ASPIRA schools have most of the ELLs in Renaissance charters), and more than 20 percent are in special education. The students at Olney include not just Latinos, but West Africans, Asians, and Haitians, he said.
He predicted that scores will start rising again because of the approach taken by ASPIRA and some of the other charter operators, which heavily relies on constant feedback for teachers.
"We’re adjusting as we’re going along. Teachers look at their lesson plans, the curriculum. … That’s how we do it — look at the data and adjust based on what the data says."
He also pointed out that the Renaissance schools were by definition the lowest-performing in the District, so the task is monumental. "It’s not easy," he said. "And it takes time."