A contentious debate about how much class size actually matters is getting some new data — and ammunition for both sides.

While former Mayor Michael Bloomberg dreamed of firing half the city’s teachers and paying the remaining superstars twice as much to teach larger classes, Mayor Bill de Blasio has argued that small classes are essential. But as classrooms have become more crowded, how much pressure should there be to reverse that trend?

A new study focusing on New York City offers some evidence for both poles of the debate: Reducing class sizes can significantly increase student learning, but those gains are often canceled out in the short run by the lower-quality teachers who wind up staffing them — and by disruptions linked to their quick hiring.

“My paper is quite supportive of the argument that class size [reduction] boosts student achievement,” explains Michael Gilraine, the study’s author, who will start this fall as a professor at New York University. But “when you reduce class sizes you’re going to have a trade-off because you need to bring in new teachers — and that might have its own independent effect.”

Gilraine’s findings come at an opportune moment for organizations like Class Size Matters and the Alliance for Quality Education, which recently filed a complaint with the State Education Department claiming the city has failed to meet required class size reduction targets.

Using data from 473 city schools, Gilraine isolated the effect of class size reductions by looking at third- through sixth-grade classes that moved just above or below the 32-student cap required for elementary grades. Classes that moved from 32 to 33 students, for instance, would have to reduce class sizes by adding a new teacher, while classes that moved from 33 to 32 students could reduce class sizes without adding a new teacher — isolating the effect of the newly added teacher.

Looking at schools near the cap between 2009 and 2014, Gilraine found that reducing class size by an average of four students produced gains in reading and math scores equivalent to roughly two and a half months of extra learning.

But there’s a big catch: The classes that shrank by bringing in a new teacher saw essentially no boost in student achievement. And since roughly 50 percent of the classes Gilraine examined depended on newly hired teachers, the overall effect of the class size reductions was cut in half.

Though Gilraine did not rigorously assess why half of these class size reductions didn’t boost student learning, he said there are a couple of likely reasons. For one, newly hired teachers may be less experienced or lower quality.

Second, there could be disruptions associated with bringing in new employees. Since class sizes were frequently lowered after the teachers union filed grievances, some reductions happened after the school year had already started — potentially disrupting classes mid-year.

The study has not been published or peer-reviewed, but Thomas Dee, director of Stanford’s Center for Education Policy Analysis, said it appeared to be rigorous. He said the findings are not entirely surprising but illustrate the importance of understanding how class size policies play out across systems, not just in the context of smaller experiments.

Dee pointed to the gold standard of class size research: a landmark randomized experiment in Tennessee conducted in the 1980s that found significantly reducing class sizes in early grades improved student learning. “That really established in people’s minds that small class sizes — though they’re expensive — are effective,” he said. But “the kinds of things that might work in a small-scale study may not scale well.”

Gilraine’s findings dovetail with research on California’s billion-dollar effort to systemically lower class sizes in the 1990s, which showed reducing class sizes led to gains in reading and math but was dampened by hiring less experienced teachers.

Dee added that one conclusion to draw from Gilraine’s study is that class size reductions may be more effective when they’re targeted — at high-need schools, for example — rather than enforced system-wide.

Leonie Haimson, executive director of Class Size Matters, said the study shows the city must do more to reduce class sizes — and do so more proactively. She emphasized that the city could avoid disruptive hiring practices, and that there’s no reason to assume hiring more teachers would reduce the overall quality of teaching in the long run.

“In a well-crafted class size reduction, you’re going to hire teachers earlier on and those teachers are going to be higher quality and stay longer and become more effective” she said. “If anything, this paper is an indictment of the current system.”