This article was originally published in The Notebook. In August 2020, The Notebook became Chalkbeat Philadelphia.
by Benjamin Herold for the Notebook and WHYY/NewsWorks
Calling it a “grave concern” that “we’re looking at data that may not be accurate,” District officials said Monday that the findings of an ongoing state investigation into possible cheating on standardized tests in more than 50 schools could throw into doubt dozens of school ratings.
“Once we have some decisions from the state, we will recalculate” the School Performance Index (SPI) results for individual schools and for the District as a whole, said Fran Newberg, deputy for accountability and educational technology .
For some schools, Newberg said, such revisions “may be very devastating.” But the overall picture of test score growth in the District, she added, should not dramatically change.
The SPI is used to make major decisions – such as which schools are turned over to charter operators due to poor performance and which are granted autonomy due to good performance, the so-called Vanguard schools. Sources told the Inquirer that 11 of the 25 Vanguard schools, or 44 percent, are among those being investigated.
Newberg’s comments came during a “Strategy, Policy and Priority” meeting of the School Reform Commission on Monday night that was heavy on technicalities. Citing persistent “confusion and mystery” around the SPI, Chairman Pedro Ramos said the SRC wanted to set aside time for a “teach-in” to establish a “foundation for future discussions.”
The SPI is the District’s system for rating and ranking schools. For each District and charter school in the city, District officials boil down either 13 indicators (for elementary and middle schools) or 17 indicators (for high schools) to a single score of 1-10, with 1 being the best.
Factors that show how much a school helps students improve on state tests – so-called “growth” criteria – are weighted most heavily. These are followed closely by data relating to the school’s raw student achievement, including what percentage of students score proficient or advanced in reading and math. Parent, student and teacher engagement and satisfaction are also considered; for high schools, career and college readiness factors are added.
But the SPI has its critics.
At Monday’s meeting, Lea School volunteer Amara Rockar said she is worried that the SPI’s heavy emphasis on PSSA results – which account for 90 percent of elementary schools’ SPI scores and 70 percent of high schools’ SPI scores – is only exacerbating the pressure on schools to cheat.
“We’ve found that kind of focus [on test scores] has had a corrupting influence on our school district,” said Rockar, citing the investigation, which is a joint state-District endeavor. In total, 53 District-run schools and three charters were deemed to have significant statistical irregularities in their PSSA results in 2009, 2010, or 2011.
Newberg said the state has not yet completed its probe into the 11 schools under the highest suspicion of cheating and that the District has not yet begun formally looking into 20 other schools where suspicious results appeared to be limited to one grade, classroom or subject.
A third tier of 22 schools, where findings could still go either way, are still being reviewed by the state.
In addition to expressing concerns about cheating and an undue emphasis on standardized test results, many in attendance at Monday’s meeting suggested that the District should give more weight to factors such as graduation and college enrollment rates for high schools.
“You get 2 percent credit for putting someone in college. That’s ridiculous,” said David Hardy, the CEO of Boys Latin Charter School in West Philadelphia.
And Richard Chapman, the CEO of Hope Charter School, questioned whether SPI is a fair measure for evaluating all schools.
“For a charter that deliberately targets low-performing … and extremely disadvantaged students, I would question the applicability of SPI,” he said.
Hope is one of three charters that District staff has recommended for non-renewal. Thomas Darden, the District’s deputy chief for strategic programs said that SPI was a “signal” that a charter needed closer scrutiny, but not a direct criterion for a renewal decision – a distinction that troubled some.
“Respectfully, I have no idea what that means,” said Andrew Sparks, an adjunct professor at Chestnut Hill College who also consults with Hope Charter.
For weeks, a subcommittee of the recently formed Great Schools Compact Committee has been looking at SPI, attempting to identify a revised formula that will satisfy both the District and the majority of its charter operators.
“There’s been no decisions made,” said the group’s chair, Lori Shorr, who is the city’s chief education officer and an executive advisor to the School Reform Commission.
But, Shorr added, “I think we will definitely get there.”