Let’s start with the good news: Over the last two decades, a rapidly increasing number of students have taken and passed Advancement Placement exams, which are often seen as helpful preparation for college.

But here’s the bad news: Many more students are also taking those courses but failing the exams. The majority of black and Hispanic test-takers don’t score a 3 or higher, which is usually needed to earn college credit.

So has the rapid expansion of AP been a net good for students and schools?

A new review of research provides a stark reminder that we simply don’t know the answer to that or a number of other important questions about AP courses, even as the program has become a more common part of the American high school experience.

Suneal Kolluri of the University of Southern California looked at over 50 studies of AP tests and classes that examine how they have expanded and whether they’ve equipped students with “college-level knowledge and skills.”

“AP is such an important element of high school for kids and teachers, and we don’t really understand how it’s impacting student experiences,” said Kolluri.

Unsurprisingly, students who score a 3 or higher on an AP exam do better in college. But, remarkably, there is virtually no research pinning down cause and effect — that is, whether taking AP courses actually helps students succeed. The association could be due to factors like a student’s high school quality or their own motivation.

The College Board, the nonprofit that oversees the AP program, points to research that shows that in four general subjects — English, math, history, and world languages — scoring a 2 is associated with boosting a student’s college grade point average in that subject. However, the effect is small, and the research can’t conclude that the GPA increase is a direct result of AP classes.

One study of Texas students in college found that the relationship between AP courses and college GPA largely disappeared once you account for a student’s overall high school academic program. The results imply that students in similar schools who took similarly hard classes do equally well in college.

Little research exists examining the trade-offs of taking an AP class rather than a community college course, a career and technical education class, or another high school class.

“The question of ‘Is that money better spent elsewhere’? We don’t have the evidence to say,” Kolluri said.

Both the College Board and individual U.S. states have prioritized increasing access to AP classes. According to the Education Commission of the States, a majority of states encourage schools to offer AP courses, some with financial incentives. Eight states specifically require districts or schools to provide access to AP classes.

Source: College Board (Graphic: Amanda Zhou/Chalkbeat)

That’s helped push participation up. Between 2001 and 2017, the total number of students taking an AP exam grew from about 820,000 to more than 2.6 million.

The success of those students varies widely. Only about 30 percent of exams taken by black students and 42 percent of exams taken by Hispanic students received a passing score in 2017, compared to 64 percent of exams taken by white students.

One explanation for that, Kolluri said, is that many students may be enrolled in low-quality AP classes. Another is that more students have not been prepared to take on the advanced material.

The existing research “is long on patterns and short on possibility,” he concludes. “While enrollment rates, AP test scores, and college attainment variables can effectively tell us where we are, more innovative investigations can better help us understand how we got here and where we might go next.”