the school improvement industry

Private groups have long tried to help turn around struggling schools. But it’s not clear if they’re doing any good.

PHOTO: Stephanie Snyder
A meeting of New York City Renewal program for struggling schools in 2014.

School improvement programs have sprouted up around the country to help turn around long-struggling schools. And the money that comes with that has also spurred the growth of external groups offering their services to schools.

But there’s little evidence on whether that school improvement industry, paid for by taxpayers, is actually boosting student learning, according to a study of 151 turnaround providers endorsed by various state agencies.  

It’s a striking finding in light of the significant role third-party groups play in supporting these efforts. But it’s not entirely surprising, considering the disappointing (albeit still debated) track record of turnaround efforts across the country, and anecdotal reports that providers with little experience emerged once the spigot of federal dollars was turned on.

There’s also a less damning explanation: Research is expensive and difficult to undertake. Programs may be successful, even if no study exists to prove it.

Still, the results raise questions about the role of third-party turnaround providers, which may continue as federal education law requires turnaround efforts in each state’s lowest performing 5 percent of schools.

This growth of outside groups has likely come because schools have had limited support to implement turnaround strategies — including from state education departments, many of whom worked with or encouraged use of external providers. In the 2012-13 school year, for instance, 34 states and two-thirds of districts implementing federal turnarounds reported contracting with external consultants to support those efforts.

“Our results suggest that states have not prioritized evidence of impact when endorsing providers to work in their schools and districts,” wrote Coby Meyers and Bryan VanGronigen of the University of Virginia.

The paper, published last month in the peer-reviewed American Journal of Education, examined 13 state websites from 2015 that listed recommended turnaround providers for struggling schools. From there, the researchers created a list of 151 state-backed providers — third-party groups that offer a variety of services to low-performing schools, including professional development, teacher recruitment, extra learning time, and improving assessment, parental involvement, and use of technology.

The researchers combed through those providers’ websites, academic research databases, and even sent emails to each organization, to see if they had research showing that their programs had led to better test scores or graduation rates.

Few did. Only 11 percent of the 151 had research backing their specific program, and then only some of those studies were focused on school turnarounds. That’s especially notable because these were all organizations recommended by state agencies, though the researchers point out, “almost no research exists on how [states] recruit, vet, endorse, or evaluate providers.”

Notably, only half of service providers reviewed claimed on their websites that their services were based in research. This meant only that the group said that its general approach was supported by research, rather than that the particular program was.

In some ways the results are no surprise, partially because of the fairly high bar being set — to be counted as having evidence, a program had to show gains based on a study that attempted to isolate cause and effect. But such studies are expensive, and researchers may be less likely to evaluate one-off programs on their own. Also, certain approaches are simply hard to evaluate. That means many programs have never been studied at all.

One of the providers that did have supportive evidence, based on a study in Ohio, was a University of Virginia program known as Partnership for Leaders in Education. That’s also where Coby Meyers, one of the researchers behind the latest study, works as chief research officer, in addition to being an associate professor at the University Virginia.

Others found to have supporting evidence included Success for All, a schoolwide turnaround program that includes efforts to strengthen professional development and curriculum; City Connects, a Boston-based community schools group, and eMINTS, which focuses on the use of technology in schools.

The researchers say it’s puzzling that so few had strong evidence since they were endorsed by state agencies. “Given how few providers had evidence of impact and how many were not research based, a key question arises: What is the rationale for endorsing providers that lack evidence of impact?” they write.

Word of mouth may play a big role in schools’ decision making. “Schools tend to contract with providers used by other schools in their own districts in the past, regardless of past performance,” according to an older study that looked at schools in Texas in the early 2000s that received federal funds to conduct what was called “comprehensive school reform.”

There’s a still great deal that’s not known about these external providers, including how much money is spent on them.

Meanwhile, turning around struggling schools remains vexing for policymakers. High-profile efforts, including by the federal government and New York City, have proven largely disappointing, though each can point to some bright spots.

The funds designated under federal law to help struggling schools are supposed to go toward “evidence-based interventions.” The standard for what counts as evidence is not especially high, though, focusing on the idea rather than the provider behind it.

Nora Gordon a professor at Georgetown who has researched such third-party groups, said their likelihood of succeeding won’t just be determined by what they plan to do, but also on local context and how they implement it.

“There could be two providers who sell intervention X and one does a good job and one does a bad job,” she said. “If you think about ESSA, they’re both going to say we’re using this research-based strategy.”

To Do

Tennessee’s new ed chief says troubleshooting testing is first priority

PHOTO: (Caiaimage/Robert Daly)

Penny Schwinn knows that ensuring a smooth testing experience for Tennessee students this spring will be her first order of business as the state’s new education chief.

Even before Gov.-elect Bill Lee announced her hiring on Thursday, she was poring over a recent report by the state’s chief investigator about what went wrong with TNReady testing last spring and figuring out her strategy for a different outcome.

“My first days will be spent talking with educators and superintendents in the field to really understand the scenario here in Tennessee,” said Schwinn, who’s been chief deputy commissioner of academics in Texas since 2016.

“I’ll approach this problem with a healthy mixture of listening and learning,” she added.

Schwinn’s experience with state assessment programs in Texas and in Delaware — where she was assistant secretary of education — is one of the strengths cited by Lee in selecting her for one of his most critical cabinet posts.

The Republican governor-elect has said that getting TNReady right is a must after three straight years of missteps in administration and scoring in Tennessee’s transition to online testing. Last year, technical disruptions interrupted so many testing days that state lawmakers passed emergency legislation ordering that poor scores couldn’t be used to penalize students, teachers, schools, or districts.

Schwinn, 36, recalls dealing with testing headaches during her first days on the job in Texas.

“We had testing disruptions. We had test booklets mailed to the wrong schools. We had answer documents in testing booklets. We had online administration failures,” she told Chalkbeat. “From that, we brought together teachers, superintendents, and experts to figure out solutions, and we had a near-perfect administration of our assessment the next year.”

What she learned in the process: the importance of tight vendor management, including setting clear expectations of what’s expected.

She plans to use the same approach in Tennessee, working closely with people in her new department and Questar Assessment, the state’s current vendor.

“Our job is to think about how to get online testing as close to perfect as possible for our students and educators, and that is going to be a major focus,” she said.

The test itself has gotten good reviews in Tennessee; it’s the online miscues that have many teachers and parents questioning the switch from paper-and-pencil exams. Schwinn sees no choice but to forge ahead online and is quick to list the benefits.

“If you think about how children learn and access information today, many are getting that information from hand-held devices and computers,” she said, “so reflecting that natural experience in our classrooms is incredibly important.”

Schwinn said computerized testing also holds promise for accommodating students with disabilities and provides for a more engaging experience for all students.

“When you look at the multiple-choice tests that we took in school and compare that to an online platform where students can watch videos, perform science experiments, do drag-and-drop and other features, students are just more engaged in the content,” she said.

“It’s a more authentic experience,” she added, “and therefore a better measure of learning.”

Schwinn plans to examine Tennessee’s overall state testing program to look for ways to reduce the number of minutes dedicated to assessment and also to elevate transparency.

She also will oversee the transition when one or more companies take over the state’s testing program beginning next school year. Former Commissioner Candice McQueen ordered a new request for proposals from vendors to provide paper testing for younger students and online testing for older ones. State officials have said they hope to award the contract by spring.

In Texas, a 2018 state audit criticized Schwinn’s handling of two major education contracts, including a no-bid special education contract that lost the state more than $2 million.

In Tennessee, an evaluation committee that includes programmatic, assessment, and technology experts will help to decide the new testing contract, and state lawmakers on the legislature’s Government Operations Committee plan to provide another layer of oversight.

Spring testing in Tennessee is scheduled to begin on April 15. You can learn more about TNReady on the state education department’s website.

Editor’s note: This story has been updated with new information about problems with the handling of two education contracts in Texas. 

Class of 2018

Some Colorado schools see big gains in grad rates. Find yours in our searchable database.

PHOTO: Courtesy of Aurora Public Schools
Aurora West College Preparatory Academy graduates of 2018. The school had a 100 percent graduation rate.

Two metro-area school districts, Westminster and Aurora, recently in the state’s crosshairs for their low-performance, posted significant increases in their graduation rates, according to 2018 numbers released Wednesday.

Westminster, a district that got off the state’s watchlist just last year, had 67.9 percent of its students graduate on time, within four years of starting high school. That was a jump of 10 percentage points from its 57.8 percent graduation rate in 2017.

District officials credit their unique model of competency-based education, which does away with grade levels and requires students prove they mastered content before moving up a level. In previous years, district officials pointed to rising graduation rates that Colorado also tracks for students who take five, six or seven years, but officials say it was bound to impact their 4-year rates as well.

“We saw an upward tick across the board this past year,” said Westminster Superintendent Pam Swanson, referring to state test results and other data also showing achievement increasing. “I think this is one more indicator.”

Swanson said the high school has also focused recently on increasing attendance, now at almost 90 percent, and increasing students’ responsibility for their own learning.

(Sam Park | Chalkbeat)

In Aurora schools, 76.5 percent of students graduated on time in 2018 — a jump of almost 9 percentage points from the 67.6 percent rate of the class of 2017.

“We’re excited these rates demonstrate momentum in our work,” Aurora Superintendent Rico Munn said.

He attributed the increased graduation rates to “better practice, better pedagogy, and better policy.”

One policy that made a difference for the district is a change in law that now allows districts to count students as graduates the year they complete their high school requirements, even if they are enrolled in one of Colorado’s programs to take college courses while doing a fifth year of high school.

According to a state report two years ago, Aurora had 65 students enrolled in this specific concurrent enrollment program who previously wouldn’t have been counted in four-year graduation rates. Only the Denver district has a larger number of such students. Aurora officials said 147 students are enrolled this year in the program.

Those students are successful, Munn said, and shouldn’t be counted against the district’s on-time graduation rates.

Aurora’s previously rising graduation rates helped it dodge corrective state action. But its improvement this year included a first: One high school, Aurora West College Preparatory Academy, had 100 percent of its seniors graduate in 2018.

The school enrolls students in grades six through 12 in northwest Aurora, the most diverse part of the district. Of the more than 1,000 students, 89 percent qualify for subsidized lunch, a measure of poverty.

“This incredible accomplishment demonstrates the strong student-focused culture we have created at Aurora West,” said Principal Taya Tselolikhina in a written statement. “When you establish high expectations and follow up with high levels of support, every student is able to shape a successful future.”

Statewide, the four-year graduation rate once again inched higher, and gaps between the graduation rate of white students and students of color again decreased. But this time, the gaps narrowed even as all student groups increased their graduation rates.

(Sam Park | Chalkbeat)

The rising trend wasn’t universal. In some metro area school districts, graduation rates fell in 2018. That includes Adams 14, the district that is now facing outside management after years of low performance.

The tiny school district of Sheridan, just southwest of Denver, saw a significant drop in graduation rates. In 2018, 64.7 percent of students graduated within four years, down from 72.7 percent of the class of 2017.

Look up four-year graduation rates for your individual school or district in our databases below.

Districts here: