Does tightening the screws on schools and teachers lead to benefits for students?

For the past couple of decades, school reform efforts have assumed that the answer is yes. Setting ambitious goals, and putting pressure on schools to reach them, would push students ahead. And past research has shown that math scores rose as more states began threatening and sanctioning schools with low test scores in the 2000s.

But a new study shows that continuing to “raise the bar” during the No Child Left Behind era only had a modest effect at best. That raises questions about whether the small gains were worth the political controversy, and what critics claim were the educational costs, of putting a greater focus on test scores.

“These results suggest that the ratcheting [up] of test-based accountability pressures alone is not enough to sustain improvements in student achievement,” conclude researchers Vivian Wong, Coady Wing, David Martin, and Anandita Krishnamachari.

Their paper, which has not been formally peer-reviewed, focuses on the several years after the federal No Child Left Behind law was signed in 2002. The law — which passed with bipartisan support but would eventually draw bipartisan ire — required states to test students annually and set goals for schools. Schools that didn’t meet them faced sanctions.

States each set their own targets using different tests. But the researchers attempted to ask the same questions of each state: How hard was it for each school to hit its goals, and how did that change between 2003 and 2011? Then, they looked at how students did on the National Assessment for Educational Progress. Did states see larger gains on the federal low-stakes test after making life tougher on schools?

In many states, it really did become harder and harder for schools to measure up. In 2008, Education Week noted that California’s school failure rate jumped from 34 to 48 percent between 2007 and 2008. In Vermont, the climb was even steeper: from 12 percent of schools failing to 37 percent.

This added pressure, the authors conclude, seemed to lead to national gains in eighth grade math and reading. But the effect was tiny: about half a point in both subjects. (For comparison’s sake, the difference in performance between white and black students in eighth grade math was 32 points on the latest test.)

“Though they find positive effects, like everyone in this literature, they are small [effects],” Tom Dee, a Stanford education professor.

That said, the gains were largest for certain disadvantaged groups: English language learners, Hispanic students, students with disabilities, and students who started at the lowest levels of performance.

There was no evidence of higher standards causing any improvements in fourth grade math or reading.

If this shows that raising the bar doesn’t do much, though, past research has shown that just having a bar can make a big difference.

In states that didn’t have accountability systems at all before No Child Left Behind, creating them led to big gains on national low-stakes math tests: 8 points in fourth grade and 5 points in eighth grade, according to a study from Dee.

Together, this research bolsters a theory known as the “accountability plateau” — that creating tougher rules boost performance, but ratcheting up the pressure leads to diminishing returns.

“It seems like when you implement an accountability system there’s an initial bump, but after that continued gains are hard to come by,” said Morgan Polikoff, a professor at the University of Southern California who has studied standards and accountability systems.

Dee was more skeptical of this idea. Schools’ goals were getting harder and harder to reach just as criticism of the law was cresting and politicians were considering changes.  

“Districts may have understood it was a nudge and a wink and it didn’t really have teeth,” he said of the law.

No Child Left Behind’s replacement, the Every Student Succeeds Act, takes a different tack. Instead of giving each state discretion in how many schools are identified as failing and requiring them to ramp up the consequences over time, the law requires each state to identify 5 percent of schools as low-performing.

The latest study suggests that might be a preferable approach if states are able to figure out better ways to help a small group of struggling schools improve. Turnaround efforts — including a prominent federal program backed by a lot of money — have often produced disappointing results.

“It remains unclear how states will implement ESSA,” write the researchers. “But the federal law will likely not succeed if performance requirements are not accompanied by additional support for educators.”