the evolution of research

Beyond the test score horse race: 5 big questions researchers are asking about charter schools

PHOTO: Dylan Peers McCoy
Christel House schools are seeking to switch charter authorizers because of administrative fees.

The latest big charter school study was sweeping in scope, looking at thousands of students in 26 states across three school years.

But the study (and lots of other research on charter schools) uses that data to answer a relatively narrow question: How do students, usually in grades 4-8, perform on math and reading tests compared to students in traditional public schools?

This could be called the “test score horse race.” Some researchers are moving beyond that, to try to understand issues like what specific charter approaches are most effective and how charter schools affect larger communities.

“A number of new research studies are beginning to investigate some more nuanced questions with regard to charters,” University of Michigan professor Brian Jacob wrote recently.

Here are a few of the big questions that some researchers are examining — and other important questions that have received little attention.

1. How do charter schools impact students beyond standardized test scores?

In education research, test scores are the coin of the realm because they are readily available. But experience and research suggest that schools affect a lot more: from attendance and behavior to college attendance, future employment, and earnings. A key question is whether test scores predicts any of those longer-term outcomes.

For answers, researchers have looked at charter schools in Boston, Florida, and Texas, as well as specific networks of schools including the Harlem Children’s Zone and the Noble Network in Chicago. Results have been mixed so far, and none of the studies has been national in scope.

2. Why are some charter schools especially effective?

We know that certain charter schools are better than others at raising test scores — but why?

There have been only a handful of studies on this, and research from New York City and Massachusetts indicate that tenets of “no excuses” charters — like extra instructional time, intensive tutoring, and frequent feedback for teachers — make a difference.

There may be other reasons for high test scores: additional resources, peer effects, harsh discipline, teaching to the test, or pushing out low-achieving students. Existing studies on those explanations have been limited in scope.

“I’d like to see more work done on how charter schools operate differently than traditional public schools — a little bit [of] getting in the black box,” Jacob said. “What are the differences in policies and practices?”

3. Why do charter schools in some places outperform charters elsewhere?

Many point to differences among authorizers, the entities — often state or local school boards — that approve and oversee charter schools. Some argue that tight controls on which schools open and the closure of low-performing schools is essential to the success of charters.

There are a handful of studies on how individual authorizers are related to charter performance, as well as the impact of charter school closures.

State policies, like the extent to which charter schools can grow, how much autonomy they have from regulations, and how much funding they get, also likely make a difference. There is remarkably little research on these questions, and what does exist is correlational.

4. How are charter schools changing over time?

“The Evolution of Charter School Quality” in Texas

Most research on charter schools functions as a snapshot that may be less useful over time. Meanwhile, the number of students attending charter schools is growing in many states, and the sector is evolving in other ways, too.

Studies in Texas and North Carolina find that charter performance improves over time, but the schools serve more advantaged students. Research in New Jersey and Arizona also suggest charters as a sector may get better as low-performing ones close down.

5. How do charter schools affect public education more broadly — including finances and segregation?

Looking only at the relative performance of charters against district schools ignores their overall impact. If charters cause traditional public schools to do better or worse through competition, for example, horse-race studies won’t capture that.

Research has found that charter schools generally have no impact or small positive effects on the test scores of students in surrounding public schools, though it is difficult to nail down cause and effect.

The financial impact of charter expansion has been studied less. Existing research has shown that charters create additional costs for districts, but that places with large charter sectors can adapt over time.

On the issue of segregation, there is evidence that charter schools in North Carolina, Pennsylvania, Indianapolis, Michigan, and Texas have exacerbated racial segregation; other studies of multiple states and using national data have found little or no impact, however.

Studies of New Orleans’ public school system, which is composed of nearly all charters, have shown that expansion of charters (as well as a number of other reforms) led to large gains in student achievement, but also caused modest increases in racial segregation in city high schools.

Jacob said he is currently working on a study of the evolution of Michigan’s charter sector. More work needs to be done, he says, to track how the expansion of choice reshapes how families select schools.

NO DEAL

Talks collapse, Denver teachers to vote on strike

PHOTO: Erica Meltzer/Chalkbeat

The Denver teachers union will hold a vote on whether to strike after months of negotiations over pay ended in deadlock.

The bargaining team of the Denver Classroom Teachers Association and officials with Denver Public Schools met all day Friday and exchanged several proposals, but they could not close a gap of more than $8 million between the two sides.

Around 10:30 p.m., Superintendent Susana Cordova said the district’s analysis found the union’s latest proposal would actually widen the money gap between the two sides, but said the district wanted to keep talking.

“It’s late, but it’s not midnight,” she said, referring to the deadline to reach an agreement.

Rob Gould, a special education teacher and member of the bargaining team, ended the discussions at that point.

“We came here tonight in good faith,” he said. “We came to correct a longstanding problem in Denver. We made movement tonight, and we’re going to talk to our teachers tomorrow.”

The room packed with red-shirted teachers erupted in cheers. Some were also crying.

Becca Hendricks, a math teacher at Emily Griffith Technical College and a member of the bargaining unit, said she felt mixed emotions at the prospect of a strike: excited at the ability to make a big change for teachers and weighed down by the responsibility.

“It doesn’t feel good,” she said. “It impacts a lot of lives.”

Cordova said she was disappointed that the union called off talks.

“We’re not at the end of the day,” she said after the meeting broke up. “We were really willing to keep talking.”

For Hendricks, it didn’t seem like there was anywhere to go.

“It became clear that the money was not a place they were going to move,” she said. “That’s a hard sticking point for us. We’ve had little to no increases for so many years, so it will take a lot of money to make up all the damage that has accrued.”

A strike requires a vote of two-thirds of the union members who cast votes, which represents about 64 percent of Denver teachers, according to the union. Teachers can join the union even on the day of the vote, which will occur on Saturday and Tuesday, but they must be members to vote.

Cordova said she would ask the state to intervene if there is a positive strike vote. The state could require the two sides to do mediation, use a fact-finder, or hold hearings to try to reach a resolution. But the state can also decline to intervene if officials don’t believe they can be productive. That intervention would delay a strike but not prevent one if the two sides still can’t agree.

The earliest that a strike could occur is Jan. 28.

Denver teachers are feeling emboldened by a surge in activism by their peers across the country that began last year and continues to build. The vote here comes as a teachers strike in Los Angeles enters its second week.

The Denver Classroom Teachers Association and Denver Public Schools are not negotiating their master contract — that deal was finalized in 2017 — but rather the ProComp system, which provides teachers bonuses for things like teaching in a high-poverty school, getting strong evaluations, having students who earn high test scores, or teaching in a high-performing school.

Denver voters approved a special tax to pay for these bonuses in 2005, which today generates around $33 million a year.

That system has been through several iterations but has been a source of frustration for many teachers because their pay was hard to understand and changed based on factors they could not control. District officials and the majority of the school board believe it is critical to keep bonuses for teachers who work in high-poverty schools as a way to retain those teachers. Turnover is a major problem in these schools and has big effects on students.

The average Denver teacher earns about $51,000 in base pay and $57,000 with incentives, according to data from the state education department and the district. That’s less than teachers in districts like Boulder Valley, Cherry Creek, and Littleton.

Both sides’ proposals moved teachers to much more predictable salary schedules that allowed for reliable raises if teachers stayed with the district and earned more education. The district proposal put an additional $20.5 million into teacher compensation, while the union’s last offer put an extra $28 million toward compensation.

The district spends about $436 million a year on teacher pay. The money for the raises would come from a combination of increased state funding and cuts to central office staff that Cordova described as deep and painful.

In addition to the total amount of money, the status of those high-poverty bonuses was a major sticking point. The district wants higher bonuses, and the union wants to put more of that money into base pay.

“To be able to bridge the gap between what is the difference in our two proposals is more than the $8 million that they were talking about because we were not willing to compromise on the need to recruit and retain teachers in our high poverty schools,” Cordova said. “We know for purposes of equity that it is so important to retain teachers in our schools that need them the most.”

But union members argued that a more reliable way to keep these teachers, who are often relatively early in their career, would be to offer them ways to more quickly increase their salary and have more stability in their economic situation. They said every other district in the region uses a reliable salary schedule, and Denver should, too.

That stance marks a major departure from some of the ideas in ProComp, among a suite of policies that have earned Denver a national reputation as an education reform hotbed over the last two decades, though both sides’ proposals met the letter of the ballot language.

Hendricks described driving a Lyft, delivering food, and tutoring to make ends meet, despite having 11 years of experience, a master’s degree, and working with at-risk students at the Emily Griffith campus. She had to move out of Denver and still has a roommate at 33 years old.

Hendricks said the union’s proposal offers higher lifetime earnings and the ability to earn raises more quickly. Cordova argues the district proposal is the stronger one for teachers, representing the largest single increase for teachers in district history and one that will give Denver teachers higher lifetime earners than those in any other metro area district.

Both sides will be trying to make their case over the next four days to teachers weighing their own compensation, the best interests of their colleagues and students, their savings accounts, and other factors in a strike vote.

More than 5,300 teachers and specialized service providers, such as social workers, psychologists, and speech language pathologists work in 147 district-managed schools. Roughly 71,000 students attend those schools.

Another 21,000 students attend Denver’s 60 charter schools.  Charter teachers are not union members, and those schools will not be affected by whatever happens next.

Cordova said she was committed to keeping schools open and providing a quality educational experience for students even if there is a strike. In Los Angeles, where teachers are also on strike, many students are watching movies and playing games during the school day. The district will offer higher pay to substitute teachers and deploy central office staff to classrooms with prepared lesson plans, she said.

Students who get subsidized lunches will still be able to eat at school.

counter-point

Four takeaways from New York City’s response to discrimination charges in specialized high schools lawsuit

PHOTO: Flickr
Brooklyn Technical is one of the city's prestigious specialized high schools.

New York City lawyers are asking a judge to allow the education department to move forward with admissions changes aimed at better integrating the city’s elite specialized high schools, saying the tweaks are not meant to discriminate against Asian students.

Instead, lawyers for the city argue the changes serve the “most disadvantaged” students, leading to “greater geographic and socioeconomic diversity” in the schools, “which may in turn increase racial diversity.”

At issue: The city’s plan to expand the Discovery program, which offers admission to students who score below the cutoff on the exam that currently stands as the sole entrance criteria to eight specialized high schools. The city also wants to change who qualifies for the program, limiting Discovery to students who attend schools where at least 60 percent of their peers are economically needy. (Previously, eligibility was based only on each students’ individual need.)

In December, Asian-American parents and organizations sued the city, claiming the reforms would discriminate against their children. They asked for a preliminary injunction, which would prevent the changes from going forward until the court case is decided — and affect the current admissions cycle already underway for eighth-graders enrolling in high school next year.

Asian students make up 62 percent of enrollment in the specialized high schools, but only 16 percent of students citywide. Meanwhile, black and Hispanic students comprise just 10 percent of enrollment in the eight schools, but 70 percent of enrollment citywide.

The city’s lawyers make a number of arguments in defense of the admissions overhaul, claiming the plaintiffs can’t bring the case because they don’t have the proper legal standing, and that it’s in the government’s interest to promote school diversity.

We already know the suit is likely to cause a delay in when students receive their high school admissions offer letters — dragging out what is already a stressful process for many families. Here are four other takeaways from the city’s response, which you can read here.

We finally know how many students will be admitted through Discovery this year, if the expansion is allowed to move forward.

Enrollment through the Discovery programs is expected to grow to 13 percent of seats at specialized high schools this summer, city lawyers wrote. That would bring the total number of students admitted through Discovery to 528, more than double last summer’s class of 252.

City leaders have previously said Discovery would be expanded gradually to eventually account for 20 percent of seats by the 2020-2021 school year. But it had been unclear until now what this year’s expansion numbers would be.

It’s uncertain whether the expansion will work as city leaders hope.

The city projects that black and Hispanic enrollment at specialized high schools would increase only modestly under the full Discovery expansion: from 9 percent to 16 percent. But city lawyers called that a “rough prediction, unlikely to definitively predict the future ethnic and racial composition of the students.”

The city’s modeling didn’t account for how many students might turn down offers to enroll in Discovery, according to court filings.

Of course, it’s possible the city is playing up the uncertainty of demographic changes for the purposes of the court fight.

Years later, a federal civil rights investigation into the specialized high schools’ admissions process is still open.

In 2012, the NAACP Legal Defense and Education Fund and other organizations filed a civil rights complaint with the U.S. Department of Education over the lack of diversity at the city’s specialized high schools.

The complaint argued that the admissions test for the sought-after schools had a disparate impact on black and Hispanic students and also took aim at the city for letting the Discovery program wither. (By 2011, only four of the high schools participated in Discovery, according to court records.)

Although the complaint hasn’t made headlines in years, it’s still under investigation, city lawyers wrote.

The Office of Civil Rights “has requested and received from DOE numerous documents and had interviewed a number of witnesses,” according to court records.

Some light was shed on how the Discovery expansion was crafted behind the scenes.

For years, Mayor Bill de Blasio promised to tackle admissions reform for the specialized high schools — but he waited until his second term to announce the proposal that’s now being challenged in court. Now we know a little more about how the current proposal was drafted.

The plan to expand Discovery and change eligibility was developed by a “decision-making group” of unnamed officials and led by Deputy Chancellor Josh Wallack, according to a statement Wallack submitted to the court. The group, in turn, recommended the changes to the schools chancellor, Richard Carranza.

“I believe the decision-making group’s recommendation was decisive in the chancellor’s decision to expand the Discovery program and adopt the revised criteria,” Wallack’s statement says.