Did you know search engines like Google are changing the way you see news? Keep Chalkbeat’s valuable education coverage at the top of your feeds by clicking here and checking the box next to our name.
After reading test scores for New York students in grades 3-5 dropped significantly last school year following big gains the year before, state education officials promised to investigate.
They shared their findings this week, saying they found no errors in the testing process but could not pinpoint the cause of the swings, deepening the mystery.
“We are pleased to report that (our) review found no errors in the calculations or processes used and confirmed that all established analysis plans and industry standards were adhered to,” JP O’Hare, spokesperson for the state Education Department, said in a statement.
Several testing experts previously said such sharp swings were unusual. In New York City, for instance,third grade reading scores fell nearly 14 percentage points last year. Fourth grade scores fell nearly 6 points, and fifth grade scores fell 13 points. Those declines essentially erased big gains made the year before, when city officials heralded the results as an indication that the city’s literacy curriculum mandate was paying off. But in both years, the trends were mirrored across the state, even in districts without such mandates.
Across grades 3-8, about half of New York City students were proficient in reading, a decline of 6 percentage points from the year before.
To investigate the results, the state Education Department worked with its technical advisory committee, a panel of experts paid by the state to oversee the testing process, as well as the test vendor, NWEA, and a third-party validation team. (Experts on the panel declined to discuss the investigation, citing a nondisclosure agreement.)
“The Department has not identified a specific reason for the changes in scores from one year to the next. It is necessary to point out that assessment results can be influenced by many factors — including but not limited to instruction, curricular materials, and administration conditions,” O’Hare wrote. “Because of this, it’s important to look at assessment results over multiple years, rather than draw conclusions from a single year’s scores.”
Parents and educators pointed to possible explanations for the unusual swings, including the transition to computer-based testing. The state previously said it conducted extensive reviews of the switch to online tests and that the change was not a factor. Still, the Education Department has not ruled out allowing New York City students to return to pencil-and-paper tests, following mounting pressure from local lawmakers to reduce screen time.
Some testing experts are still trying to understand what happened.
Andrew Ho, a testing expert at Harvard’s Graduate School of Education, said that while swings like New York’s are striking, they’re increasingly common.
“Since the pandemic, we’ve seen more anomalies like this. Trends depend on changes in population, motivation, and administration,” Ho, who is on the state’s technical advisory committee but was speaking in a personal capacity, wrote in an email. “If tested populations change more, or kids and teachers care more, or administration conditions change, we see swings that don’t represent real changes in average achievement.”
He added: “The testing industry needs to improve its equating procedures or risk public trust.”
To help the public better understand the results and improve transparency, EdTrust-New York, a statewide education policy and advocacy group, called for an independent audit of the last three years of the state’s reading tests after officials released their findings without a public presentation or detailed analysis of the results.
Emily Oster, a Brown University education economist who closely follows test scores nationwide, has been vocal about her skepticism of the anomalous scores. She continues to question why reading scores for grades 3-5 plunged after soaring the year before, while grades 6-8 and math scores showed no similar changes.
“The changes are enormous, implying an increase of about 2/3 of a year of learning, and then a subsequent loss for the same children year-on-year,” Oster wrote in an email to Chalkbeat. “It’s very unlikely that these reflect changes in learning, or curricular changes. They happened at the same level for basically every racial group, both disadvantaged and advantaged students, and across the entire state. This is just really difficult to explain with learning gains.”
Oster raised the possibility that something may have gone wrong in calibrating test questions from year to year — a process that allows test makers to compare an individual’s score to those of peers — but acknowledged she had no firm evidence.
State Education Department officials did not provide further information on the investigation’s findings or on Oster’s theory.
Amy Zimmer is the bureau chief for Chalkbeat New York. Contact Amy at azimmer@chalkbeat.org.



