Beyond the test score horse race: 5 big questions researchers are asking about charter schools

The latest big charter school study was sweeping in scope, looking at thousands of students in 26 states across three school years.

But the study (and lots of other research on charter schools) uses that data to answer a relatively narrow question: How do students, usually in grades 4-8, perform on math and reading tests compared to students in traditional public schools?

This could be called the “test score horse race.” Some researchers are moving beyond that, to try to understand issues like what specific charter approaches are most effective and how charter schools affect larger communities.

“A number of new research studies are beginning to investigate some more nuanced questions with regard to charters,” University of Michigan professor Brian Jacob wrote recently.

Here are a few of the big questions that some researchers are examining — and other important questions that have received little attention.

1. How do charter schools impact students beyond standardized test scores?

In education research, test scores are the coin of the realm because they are readily available. But experience and research suggest that schools affect a lot more: from attendance and behavior to college attendance, future employment, and earnings. A key question is whether test scores predicts any of those longer-term outcomes.

For answers, researchers have looked at charter schools in Boston, Florida, and Texas, as well as specific networks of schools including the Harlem Children’s Zone and the Noble Network in Chicago. Results have been mixed so far, and none of the studies has been national in scope.

2. Why are some charter schools especially effective?

We know that certain charter schools are better than others at raising test scores — but why?

There have been only a handful of studies on this, and research from New York City and Massachusetts indicate that tenets of “no excuses” charters — like extra instructional time, intensive tutoring, and frequent feedback for teachers — make a difference.

There may be other reasons for high test scores: additional resources, peer effects, harsh discipline, teaching to the test, or pushing out low-achieving students. Existing studies on those explanations have been limited in scope.

“I’d like to see more work done on how charter schools operate differently than traditional public schools — a little bit [of] getting in the black box,” Jacob said. “What are the differences in policies and practices?”

3. Why do charter schools in some places outperform charters elsewhere?

Many point to differences among authorizers, the entities — often state or local school boards — that approve and oversee charter schools. Some argue that tight controls on which schools open and the closure of low-performing schools is essential to the success of charters.

There are a handful of studies on how individual authorizers are related to charter performance, as well as the impact of charter school closures.

State policies, like the extent to which charter schools can grow, how much autonomy they have from regulations, and how much funding they get, also likely make a difference. There is remarkably little research on these questions, and what does exist is correlational.

4. How are charter schools changing over time?

“The Evolution of Charter School Quality” in Texas

Most research on charter schools functions as a snapshot that may be less useful over time. Meanwhile, the number of students attending charter schools is growing in many states, and the sector is evolving in other ways, too.

Studies in Texas and North Carolina find that charter performance improves over time, but the schools serve more advantaged students. Research in New Jersey and Arizona also suggest charters as a sector may get better as low-performing ones close down.

5. How do charter schools affect public education more broadly — including finances and segregation?

Looking only at the relative performance of charters against district schools ignores their overall impact. If charters cause traditional public schools to do better or worse through competition, for example, horse-race studies won’t capture that.

Research has found that charter schools generally have no impact or small positive effects on the test scores of students in surrounding public schools, though it is difficult to nail down cause and effect.

The financial impact of charter expansion has been studied less. Existing research has shown that charters create additional costs for districts, but that places with large charter sectors can adapt over time.

On the issue of segregation, there is evidence that charter schools in North Carolina, Pennsylvania, Indianapolis, Michigan, and Texas have exacerbated racial segregation; other studies of multiple states and using national data have found little or no impact, however.

Studies of New Orleans’ public school system, which is composed of nearly all charters, have shown that expansion of charters (as well as a number of other reforms) led to large gains in student achievement, but also caused modest increases in racial segregation in city high schools.

Jacob said he is currently working on a study of the evolution of Michigan’s charter sector. More work needs to be done, he says, to track how the expansion of choice reshapes how families select schools.