Abolish middle school? Not so fast, new study says.

The push to combine elementary and middle schools into K-8 schools has seemed like a heartening example of policymakers making decisions based on hard evidence.

Rigorous studies have suggested that scrapping traditional middle schools is good for students. And some districts like Boston have moved to merge schools, trying to eliminate some of the elements of middle school that make it miserable for many tweens.

New research says, hold on a second.

It suggests that past studies have overstated the benefits of K-8 schools, and offers a warning to districts moving to eliminate middle schools — as well as a parable of how complicated it can be to make decisions based on the shifting findings of education research.

The paper, published earlier this month in the peer-reviewed Journal of Urban Economics, uses school closures and shifting school zone boundaries in one district to isolate the effects of attending a K-8 school versus attending an elementary school until fifth grade and then a separate middle school.

Like past research, the study finds that transitioning to a middle school leads to a dip in test scores in math. But students in grades three through five do better at a stand-alone elementary school, making up for that sixth-grade dip. By eighth grade, attending a K-8 school has no effect in math.

The results in reading were even more surprising: students in separate middle schools made larger gains in seventh and eighth grade, and ended middle school with higher scores than their peers in K-8 schools.

“The adverse effects for elementary students in K-8 schools combined with the lack of long-term adverse effects for students attending separate middle schools does not provide support for K-8 configuration,” researchers Kai Hong, Ron Zimmer, and John Engberg write. “In fact, our results provide some evidence against K-8 schools as a policy.”

Other studies have come to a different conclusion. Research on Canada, Florida, and a number of studies in New York City point to benefits of K-8 schools, including in test scores, attendance, and even high school performance in one study. This has prompted headlines like “Why Middle School Should Be Abolished.”

It’s not entirely clear why the latest results are different. It could be that, through luck or other reasons, certain districts have better or worse K-8 schools. The authors of the latest study point to wonky methodological issues, arguing that past research isn’t able to capture the negative effects of K–8 schools on elementary students.

On the other hand, the recent paper is one study of just one (anonymous) district, so extrapolating from the results is a dicey proposition — particularly when the weight of the research is on the other side.

Amy Ellen Schwartz, a professor at the Maxwell School at Syracuse University, praised the latest research but noted an important limitation: it relies on the assumption that the redrawing of school boundaries is essentially random.

Schwartz, who has conducted some of the past research pointing to benefits of K-8 schools, says it’s important for policymakers to really consider the pros and cons of middle schools. Separate schools might be ideal for policymakers who want to emphasize school choice, but others “might particularly like a K-8 [school] in a world where kids have unstable lives and the stability might be good for them,” she said.

“What is important is to try to be a little more nuanced on this,” she said.