the evolution of research

Beyond the test score horse race: 5 big questions researchers are asking about charter schools

PHOTO: Dylan Peers McCoy

The latest big charter school study was sweeping in scope, looking at thousands of students in 26 states across three school years.

But the study (and lots of other research on charter schools) uses that data to answer a relatively narrow question: How do students, usually in grades 4-8, perform on math and reading tests compared to students in traditional public schools?

This could be called the “test score horse race.” Some researchers are moving beyond that, to try to understand issues like what specific charter approaches are most effective and how charter schools affect larger communities.

“A number of new research studies are beginning to investigate some more nuanced questions with regard to charters,” University of Michigan professor Brian Jacob wrote recently.

Here are a few of the big questions that some researchers are examining — and other important questions that have received little attention.

1. How do charter schools impact students beyond standardized test scores?

In education research, test scores are the coin of the realm because they are readily available. But experience and research suggest that schools affect a lot more: from attendance and behavior to college attendance, future employment, and earnings. A key question is whether test scores predicts any of those longer-term outcomes.

For answers, researchers have looked at charter schools in Boston, Florida, and Texas, as well as specific networks of schools including the Harlem Children’s Zone and the Noble Network in Chicago. Results have been mixed so far, and none of the studies has been national in scope.

2. Why are some charter schools especially effective?

We know that certain charter schools are better than others at raising test scores — but why?

There have been only a handful of studies on this, and research from New York City and Massachusetts indicate that tenets of “no excuses” charters — like extra instructional time, intensive tutoring, and frequent feedback for teachers — make a difference.

There may be other reasons for high test scores: additional resources, peer effects, harsh discipline, teaching to the test, or pushing out low-achieving students. Existing studies on those explanations have been limited in scope.

“I’d like to see more work done on how charter schools operate differently than traditional public schools — a little bit [of] getting in the black box,” Jacob said. “What are the differences in policies and practices?”

3. Why do charter schools in some places outperform charters elsewhere?

Many point to differences among authorizers, the entities — often state or local school boards — that approve and oversee charter schools. Some argue that tight controls on which schools open and the closure of low-performing schools is essential to the success of charters.

There are a handful of studies on how individual authorizers are related to charter performance, as well as the impact of charter school closures.

State policies, like the extent to which charter schools can grow, how much autonomy they have from regulations, and how much funding they get, also likely make a difference. There is remarkably little research on these questions, and what does exist is correlational.

4. How are charter schools changing over time?

“The Evolution of Charter School Quality” in Texas

Most research on charter schools functions as a snapshot that may be less useful over time. Meanwhile, the number of students attending charter schools is growing in many states, and the sector is evolving in other ways, too.

Studies in Texas and North Carolina find that charter performance improves over time, but the schools serve more advantaged students. Research in New Jersey and Arizona also suggest charters as a sector may get better as low-performing ones close down.

5. How do charter schools affect public education more broadly — including finances and segregation?

Looking only at the relative performance of charters against district schools ignores their overall impact. If charters cause traditional public schools to do better or worse through competition, for example, horse-race studies won’t capture that.

Research has found that charter schools generally have no impact or small positive effects on the test scores of students in surrounding public schools, though it is difficult to nail down cause and effect.

The financial impact of charter expansion has been studied less. Existing research has shown that charters create additional costs for districts, but that places with large charter sectors can adapt over time.

On the issue of segregation, there is evidence that charter schools in North Carolina, Pennsylvania, Indianapolis, Michigan, and Texas have exacerbated racial segregation; other studies of multiple states and using national data have found little or no impact, however.

Studies of New Orleans’ public school system, which is composed of nearly all charters, have shown that expansion of charters (as well as a number of other reforms) led to large gains in student achievement, but also caused modest increases in racial segregation in city high schools.

Jacob said he is currently working on a study of the evolution of Michigan’s charter sector. More work needs to be done, he says, to track how the expansion of choice reshapes how families select schools.

The New Chancellor

Tell us: What should the new chancellor, Richard Carranza, know about New York City schools?

PHOTO: Christina Veiga
A student at P.S. 69 Journey Prep in the Bronx paints a picture. The school uses a Reggio Emilia approach and is in the city's Showcase Schools program.

In a few short weeks, Richard Carranza will take over the nation’s largest school system as chancellor of New York City’s public schools.

Carranza, who has never before worked east of the Mississippi, will have to get up to speed quickly on a new city with unfamiliar challenges. The best people to guide him in this endeavor: New Yorkers who understand the city in its complexity.

So we want to hear from you: What does Carranza need to know about the city, its schools, and you to help him as he gets started April 2. Please fill out the survey below; we’ll collect your responses and share them with our readers and Carranza himself.

The deadline is March 23.

buses or bust?

Mayor Duggan says bus plan encourages cooperation. Detroit school board committee wants more details.

PHOTO: Denver Post file
Fourth-graders Kintan Surghani, left, and Rachel Anderson laugh out the school bus window at Mitchell Elementary School in Golden.

Detroit’s school superintendent is asking for more information about the mayor’s initiative to create a joint bus route for charter and district students after realizing the costs could be higher than the district anticipated.

District Superintendent Nikolai Vitti told a school board subcommittee Friday that he thought the original cost to the district was estimated to be around $25,000 total. Instead, he said it could cost the district roughly between $75,000 and a maximum of $125,000 for their five schools on the loop.

“I think there was a misunderstanding….” Vitti said. “I think this needs a deeper review…The understanding was that it would be $25,000 for all schools. Now, there are ongoing conversations about it being $15,000 to $25,000 for each individual school.”

The bus loop connecting charter and district schools was announced earlier this month by Mayor Mike Duggan as a way to draw kids back from the suburbs.

Duggan’s bus loop proposal is based on one that operates in Denver that would travel a circuit in certain neighborhoods, picking up students on designated street corners and dropping them off at both district and charter schools.

The bus routes — which Duggan said would be funded by philanthropy, the schools and the city — could even service afterschool programs that the schools on the bus route could work together to create.

In concept, the finance committee was not opposed to the idea. But despite two-thirds of the cost being covered and splitting the remaining third with charters, they were worried enough about the increased costs that they voted not to recommend approval of the agreement to the full board.  

Vitti said when he saw the draft plan, the higher price made him question whether the loop would be worth it.

“If it was $25,000, it would be an easier decision,” he said.

To better understand the costs and benefits and to ultimately decide, Vitti said he needs more data, which will take a few weeks. 

Alexis Wiley, Duggan’s chief of staff, said the district’s hesitation was a sign they were performing their due diligence before agreeing to the plan.

“I’m not at all deterred by this,” Wiley said. She said the district, charters, and city officials have met twice, and are “working in the same direction, so that we eliminate as many barriers as we can.”

Duggan told a crowd earlier this month at the State of the City address that the bus loop was an effort to grab the city’s children – some 32,500 – back from suburban schools.

Transportation is often cited as one of the reasons children leave the city’s schools and go to other districts, and charter leaders have said they support the bus loop because they believe it will make it easier for students to attend their schools.

But some board members had doubts that the bus loop would be enough to bring those kids back, and were concerned about giving charters an advantage in their competition against the district to increase enrollment.

“I don’t know if transportation would be why these parents send their kids outside of the district,” Angelique Peterson-Mayberry said. “If we could find out some of the reasons why, it would add to the validity” of implementing the bus loop.

Board member LaMar Lemmons echoed other members’ concerns on the impact of the transportation plan, and said many parents left the district because of the poor quality of schools under emergency management, not transportation.

“All those years in emergency management, that drove parents to seek alternatives, as well as charters,” he said. “I’m hesitant to form an unholy alliance with the charters for something like this.”