Alabama’s high school graduation rate was one of the lowest in the country in 2011. Today, it’s one of the highest.

Over that same period, though, Alabama students have continued to perform among the worst in the nation on federal math and reading tests.

That leaves the state with a jarring disconnect between its students’ academic skills and the share of diplomas it hands out. And while Alabama’s numbers are outliers, that disconnect exists in many other states.

States with low test scores don’t necessarily have low graduation rates, and vice versa, data released last week for the class of 2017 shows. And state test scores are less pegged to graduation rates than they were several years ago, according to a Chalkbeat analysis.

It’s the latest indication that the nation’s graduation rate gains may have more to do with changes in graduation standards than with how much students are learning. The U.S. graduation rate rose from 79 percent to 84.6 percent from 2011 to 2017, even as test scores stayed largely flat.

“The high school graduation rate numbers have been going up and up and up, which I do think is a good outcome,” said Anne Hyslop, a former Department of Education official now at the Alliance for Excellent Education, a nonprofit that focuses on improving high schools. “But it also calls into question whether all of those diplomas mean the same thing, whether they are as meaningful a credential as it once was.”

Chalkbeat compared each state’s graduation rate in 2017 and its students’ scores on the federal eighth grade NAEP exams given in 2013. (Twelfth-grade NAEP scores aren’t available for most states.)

Some states, like Massachusetts and New Jersey, have relatively high test scores and graduation rates. Others, like New Mexico and Louisiana, are low on both counts. But many others have a mismatch. Washington and Colorado, for example, have high test scores and low graduation rates, while in Texas and West Virginia, students score poorly on tests but have among the country’s highest graduation rates.

Overall, there is a correlation between states’ test scores and graduation rates — but it’s a modest one.

Why is there this disconnect?

For one, states simply have different graduation standards. Some — though increasingly few — require students to pass a set of exams to graduate, for example.

The numbers may also miss something about the quality of a state’s high schools or students skills. Eighth-grade scores don’t account for what happens in later grades, and test scores aren’t perfect gauges of students’ abilities.

Whatever the reason, the disconnect is growing larger. Six years ago, eighth-grade NAEP scores were more closely tied to a state’s graduation rate. That’s true for both math and reading scores.

In 2011, high-NAEP states had graduation rates nearly 15 percentage points higher on average than those with low scores. Last year, the difference was less than 10 percentage points.

What explains this? In general, graduation rates are rising fast, and states that started with low graduation rates had more room to grow.

Some of those gains might be because schools are juking their graduation numbers.

“If states and local districts allow all sorts of other factors to come into play (credit recovery, lower standards, outright fraud), then over time high school graduation rates will grow more removed from academic achievement,” Mark Dynarski, a researcher who has questioned rising graduation rates, wrote in an email.

A series of stories from NPR in 2015 found that fast-track credit recovery programs, which allow students who fall behind to graduate on time, were one of several factors behind rising graduation rates between 2001 and 2013. Others included schools helping students stay on track early in their high school careers and schools gaming the numbers by miscategorizing dropouts.

Audits by the U.S. Department of Education found that both Alabama and California had inflated their graduation rates by improperly counting some students.

The news isn’t all bad, though. More kids staying in high school and earning diplomas may be beneficial for them as individuals and for society as a whole. And even flat 12th-grade test scores might be an encouraging sign, since students who might have dropped out were taking those tests.

Hyslop said it’s not clear where all this leaves policymakers. One potential response would be to raise the bar to graduate through high school exit exams. But research has shown that this approach has few clear benefits while increasing dropout rates, particularly among black and Hispanic students.

“Denying someone a high school diploma is a really weighty decision and one that we have been reticent to make,” she said.

Graphic by Gabrielle LaMarr LeMee.