schools of the future

As ed reformers urge a ‘big bet’ on personalized learning, research points to potential rewards — and risks

Philanthropists and school leaders need to make a “big bet” on dramatically reshaping schools, according to the leaders behind last week’s major education conference.

Re-imagining learning, schools of tomorrow, personalized learning, jobs of the future — these were the watchwords at the New Schools Venture Fund Summit. Ideas about how to reinvent schooling were more prominent than even hot-button topics like school integration or vouchers.

“The world has changed dramatically … and our schools have struggled to keep up,” said New Schools CEO Stacey Childress at the summit’s opening session. “We just think it’s time to update the way schools work so they better prepare students for success in today’s world.”

The solution, according to some, is a focus on innovative school models — particularly ones that use technology to “personalize” teaching based on students’ needs, desires, strengths, and weaknesses.

The gathering was underwritten by deep-pocketed funders known for backing technology-based education initiatives, including the Gates Foundation and the relatively new Chan Zuckerberg Initiative. (Gates is a supporter of Chalkbeat, as are the Walton Family Foundation and the Carnegie Corporation, which were also major sponsors of the event.)

Advocates are extremely optimistic about investing in this new breed of schools. A report released by New Schools at the end of last year made an aggressive call to direct $4 billion in philanthropy toward creating, studying, and promoting this sort of innovation. Doing so, the authors write, will produce an estimated 200 to 500 percent return on investment.

It’s a remarkable claim, but perhaps an overly rosy one: the author of the study that New Schools relied on to make such a strong assertion says it’s likely inflated. And although some studies point to the benefits of specific technology programs, the research about whole schools using these approaches remains in its infancy.

As the ideas gain prominence, skeptics fear that the push amounts to the latest fad in education.

“What I see … is people imagining that if we just design the school with new models we will be able to satisfy the needs of the future,” said Ben Riley, head of the group Deans for Impact and former New Schools staff member. “The graveyard of people thinking they could successfully predict the future and then finding out that they were wrong about that has a lot of tombstones.”

The appeal of innovative schools and a personalized approach

Amidst a sea of buzzwords, it can be hard to define school innovation or personalized learning. (Some even argue that trying to pin down a clear meaning is misguided.)

Roughly speaking, though, the idea epitomized at the New Schools summit focuses on expanding technology in schools to better tailor teaching to specific students’ needs and desires.

The New Schools report points to several traits of these schools: maximizing “time, pace, instructional methods and outside experiences”; using “an expanded definition of student success”; ensuring students “feel ownership of their learning”; more frequent use of technology; and ensuring students build trusting relationships.

Advocates often point to Summit Public Schools, a charter school network in California that embraces a technology-infused model. A favorable case study authored by David Osborne of the reform-oriented Progressive Policy Institute calls them “schools of the future.”

Students at Summit spend about 16 hours a week — half of their school time — learning via computer. As Osborne describes it: “Teachers were there to answer questions, make suggestions when kids got stuck, and check their progress, but students were in charge of their own learning. They worked at their own pace, and when they felt they had mastered a concept, they took a 10-question assessment. If they could answer eight of the questions correctly, they checked that off and moved on to the next topic.”

Supporters of that approach highlight an analysis released by the RAND Corporation and the Gates Foundation. The study, which looked at schools funded by Gates because of their “promising approaches to personalized learning,” found that students made much larger gains on standardized tests compared to peers with similar characteristics attending different schools.

There is broader evidence that specific technology programs can improve achievement. Studies of computer programs designed to help students with algebra and early literacy, as well as the math program used by the Rocketship charter network, have shown benefits for students.

“The findings in general are very positive, especially for math,” said Andre Nickow, a Northwestern graduate student who worked on an analysis of technology-based personalized learning that was recently presented at a research conference.

Proponents also say the push for personalized learning is based on a deeper understanding of how kids learn — and how they can benefit from individualized instruction.

“The evidence base for the benefits of 1:1 mastery-based instruction is quite strong,” said Debbie Veney, a spokesperson for New Schools, pointing to a 1984 analysis of individual tutoring by prominent educational psychologist Benjamin Bloom, creator of the well-known Bloom’s taxonomy.

But providing each student a personal tutor is prohibitively expensive; the idea is to use technology to provide that personalization at a fraction of the cost.

“The innovative schools we and others support are working to find ways to produce similar academic results for students by using a mix of instructional approaches,” Veney said. “The early results of schools working on this challenge are quite promising, including the schools in the RAND study.”

Students at University Prep, a Denver elementary charter school, work on a computer-based assignment .
PHOTO: Marc Piscotty

The concerns about existing research

Indeed, that RAND study of innovative schools using personalized learning is often cited in these conversations. But there are a few reasons to view the findings cautiously.

The first is how New Schools frames the results in its report. Based on the RAND analysis, New Schools estimates a 200–500 percent return on the $4 billion the group argues philanthropists should invest over 10 years. But John Pane, author of the RAND study, disputes those projections.

“I was not really keen on some of the leaps that they were making based off of our data,” he told Chalkbeat. Pane himself estimated a potential return on investment and said he found “vastly smaller numbers” than New Schools did. (He declined to share his exact figures because his results have not gone through peer review yet.)

Veney said that the method in the report — converting effects on test scores into days of learning — is common in education research, including the CREDO charter studies.

The report has another major limitation, which Pane acknowledges: the schools in the study had to go through a rigorous application process in order to receive funding. That means the academic gains described in the study may not be the result of the schools’ use of personalized learning per se, but simply the possibility that only high-performing schools were examined.

“For me, the headline was good schools do good things,” said Alex Hernandez, a partner at the Charter School Growth Fund, which invests in new school models.

Regarding the two leading “pioneers” identified by New Schools, the research is also limited. Studies of the New Classrooms model, a personalized learning program that began in New York City known as Teach to One, have found mixed results. There does not appear to be any external empirical research on Summit Public Schools, though an internal analysis says students in the schools make large gains on standardized tests.

Skeptics say that the theory behind personalized learning has some inherent flaws, too.

For instance, Diane Tavenner, Summit’s CEO, previously said, “Look at the economy: it’s not about concrete knowledge, it’s about higher-order thinking skills, and the ability to perpetually learn and grow.”

Sentiments like that worry Ben Riley, who argues that the best way to train students for the jobs of tomorrow is to “provide them with a really rich, broad comprehensive education that gives them the knowledge base that allows them to adapt to those new jobs.”

“If there’s one thing that cognitive science has shown over and over again, it’s that our ability to understand new ideas depends on what ideas we already know,” said Riley, whose group released a report compiling research on the science of learning.

Meanwhile, unmentioned in New Schools’ report is research on fully virtual schools, in which instruction is delivered exclusively online. Although supporters might not categorize this approach as personalized learning, the schools are certainly different from traditional schools in their reliance on technology. And yet the results to date have been abysmal — students have generally seen large drops in test scores relative to those in brick-and-mortar schools. This suggests that expanding technology in education can come with significant risks.

If it fails, move on — but who is left behind?

New Schools acknowledges that these innovative schools aren’t a sure thing. (It is calling the plan a “big bet,” after all.) But trying out new structures for schools is necessary to prepare more students to succeed in college, especially the black, Hispanic, and low-income students who haven’t been served well by typical schools, they argue.

“If after several years this approach isn’t living up to the potential we imagine, let’s change course,” the report states. “But let’s also evaluate every other idea for how to direct the $20 billion in education philanthropy over the next 10 years based on concrete estimates of the improved outcomes we should expect for students.”

As schools experiment, “We will fail,” said Derwin Sisnett, cofounder of the Gestalt Community Schools charter network, at the summit.

But critics of the approach say such failure could lead to collateral damage.

“Should philanthropists scrap the initiative after five years, 1,700 schools will be left to deal with the aftermath and lost funding despite the many constraints they face,” wrote Jeffrey Snyder of Cleveland State University in a critical review of the New Schools report for the National Education Policy Center, which is partially funded by teachers unions.

Veney of New Schools said the group’s vision is to use philanthropic funds temporarily to jumpstart innovation. “Our proposal is to support educators who want to make the shifts with the resources they need to do it well,” she said, “and to ensure that the designs and practices they are adopting are sustainable with public funding after the first few years.”

Snyder, though, writes that the risks extend beyond the financial considerations. “Especially concerning should be the potential for this philanthropic shift to continue (and exacerbate) reform churn — the process where schools move quickly among reforms, never allowing any to take root.”

Voucher debate update

Students given vouchers in Louisiana do worse than their peers. But new research shows some kids catch up

PHOTO: Grace Tatter

Past research on Louisiana’s school voucher program came to a bleak conclusion: students who used the program to transfer to a private school saw their test scores plummet.

A new study complicates that narrative, finding some good — or at least, less bad — news about the closely watched program.

The research shows that, for students who received a voucher at the middle or end of elementary school, there were no statistically significant effects on their math or reading test scores by the third year in the program. That’s a boon for voucher advocates who have argued against judging a program by its initial impacts.

This “is an initial study of a very long-term question: namely, can government create a level playing field for all types of schools so that the best of all types of schools are available to the most disadvantaged students?” John White, Louisiana’s schools superintendent, told Chalkbeat. “I think this study shows that we are on our way to making that happen.”

Still, other aspects of the study suggest that the program continues to have negative effects, often large ones, on some students, specifically those in early grades.

The findings come as U.S. Education Secretary Betsy DeVos has promised a major federal push to expand private school choice. The Louisiana Scholarship Program, a statewide initiative that uses public funds to pay private school tuition for certain low-income students, is the seventh-largest voucher program in the country, serving over 7,000 students.

The research is a follow-up to an earlier study conducted by Jonathan Mills and Patrick Wolf of the University of Arkansas and has not been formally peer-reviewed. The study was funded by the Smith Richardson Foundation.

Mills and Wolf compare students who won the chance to participate in Louisiana’s program against those who applied for a voucher but lost the lottery. That means the researchers can be confident that differences between the two groups are caused by receiving the voucher.

For a number of technical reasons, the study focuses on just 514 older elementary school students who won a spot in a private school and remained there for three years, though they amount to only a small subset of the students using a voucher in the state.

The researchers find that by year three, the impacts of the program were slightly positive in English, moderately negative in math and science, and highly negative in social studies — but none of these results were statistically significant. They do find statistically significant positive effects in English for students who started out low performing in the subject.

White said the improvements in math and English compared to earlier years were reason for optimism.

The study also examines a larger sample that also included younger students. Here the results are decisively negative: Receiving a voucher led to large, statistically significant test score drops in both math and social studies.

But the authors caution that they are less confident in these results than in the findings exclusively for older students because they don’t have baseline test scores for such young students.

Why did the program seem to help students more when they remained for three years? White credits the threat of Louisiana’s test-based accountability rules, which are more stringent than most other states’ rules for private school choice programs. Critics have argued that these rules deter top-notch private schools from participating.

“The important finding is that, given accountability, schools improve — that includes private schools, as well as public schools,” White said.

The researchers are more skeptical of this explanation. They suggest that the improvement may have been the result of students for whom the program was not working returning to public schools, and adjustments made by private schools over time, including aligning curriculum to the state exam.

“It is possible that the private schools ignored those [potential] sanctions in year 1 and started taking them seriously in years 2 and 3, leading to the pattern of effects we observe, but it seems unlikely that they would place themselves in such a disadvantaged position from the start,” Mills and Wolf wrote.

Another study released Monday, of Indiana’s voucher program, showed that students in the program saw math achievement drop in comparison to public school students, but those who remained in private school for four years caught up in math and made gains in English.

Voucher debate update

First study of Indiana’s voucher program — the country’s largest — finds it hurts kids’ math skills at first, but not over time

PHOTO: Scott Elliott
Cardinal Ritter High School has one of the highest percentages of students paying tuition with state-funded vouchers in Indiana.

A new analysis of Indiana’s school voucher program offers something for both sides in the heated debate about whether public money should be used to fund private school tuition.

The study, obtained by Chalkbeat, shows that students using a voucher saw math achievement fall on average, though students who remained in private school for four years improved to match or outperform public school students in math and English.

The results amount to a Rorschach test for advocates on either side of the issue.

“At the end of four years, English scores are slightly above where students started and math scores are statistically the same — so the trend line is heading the right way,” said Robert Enlow, the president of EdChoice, an Indianapolis-based group that backs voucher and tax credit programs.

“Indiana diverted millions of dollars for years from public schools to private school vouchers, resulting in negative or negligible results for student outcomes,” said American Federation of Teachers president Randi Weingarten in a statement. “This latest study of vouchers should be yet another red flag to [U.S. Education Secretary Betsy] DeVos that she is going down the wrong path.”

The report was provided by researchers Mark Berends and Joseph Waddington after Chalkbeat obtained an earlier version of the study through a public records request to the Indiana Department of Education.

The study has been submitted to a peer-reviewed journal but has not yet been published. (An initial description posted online —  later taken down — has drawn wide media attention, and the researchers also previously presented preliminary findings to separate gatherings of academics and school choice advocates.)

The authors declined to comment on the results but criticized Chalkbeat’s decision to release them.

“It does a disservice to social scientists who want to make sure their research passes peer review before being publicly released,” Berends of the University of Notre Dame and Waddington of the University of Kentucky wrote in an email.

Chalkbeat is publishing the research because it is on a matter of pressing public concern — whether low-income students given public dollars to attend private schools learn more than they would in public schools, as the Trump administration promises to push for more voucher programs like Indiana’s.

“In Indiana, we’ve seen some of the best pro-parent and pro-student legislation enacted in the country,” DeVos recently said, referring to the state’s private and charter school initiatives. Her former advocacy group, American Federation for Children, heavily backed Indiana’s school voucher program while she was the group’s chairperson.

Recent research has found that voucher programs can lead to drops in test scores, but some studies like those in D.C. and Louisiana only examine the first one or two years of the program. The latest analysis, the first statewide study of Indiana’s program, looks at four years of data — and offers evidence that judging programs by short-term results may be unfair.

A spokesperson for the Indiana Department of Education declined to comment on the results, saying the department would do so once the study completes the peer-review process.

Voucher students lose ground in math, but those who stick around see gains in English

The paper examines the first four years — from the 2011-12 school year to 2014-15 — of Indiana’s private school voucher program, the largest in the country.

The initiative was championed by former governor Mitch Daniels and expanded to include middle-class families under Mike Pence, now vice president. In Indiana, participating schools are largely religious, and unlike in some school choice programs, students take state tests and schools can be barred from accepting new voucher students for poor academic performance.

The researchers focus on low-income students in the middle or end of elementary school who switched from public schools to private schools using a voucher, and compare them to similar students who remained in public school.

Relative to low-income students in public schools, those whose family elected to use a voucher were more likely to be female, Latino, and an English-language learner, and less likely to be black or have a disability. The voucher students also had slightly higher initial test scores, though still below the state average.

Student achievement in English over time

Compared to other private school students in the state, voucher recipients were more racially diverse, more likely to be low-income, and had significantly lower test scores.

The study estimates that receiving a voucher led to moderate decreases in math test scores overall. Students who participated in the program for four consecutive years initially saw a drop, but by year four they had caught back up to their public school counterparts.

When looking at English scores, the data suggest that there was no impact, good or bad, of receiving a voucher on average. However, the subset of students who remained in the program for all four years appeared to be doing moderately better in English than those in public schools.

In contrast to students who stuck with the program for several years, those who eventually left private schools saw large decreases in achievement while they were using a voucher.

There were not major differences across students by ethnicity or gender. But students with disabilities saw significant decreases in English test scores, while Catholic schools improved English achievement.

(The study was funded in part by the Walton Foundation, which is a supporter of Chalkbeat. EdChoice is also a Chalkbeat funder. Learn more about our funding here.)

Study validates advocates’ argument not to rush to judgment based on early years

The analysis of the program joins recent research showing that voucher programs can hurt student achievement. But the study’s finding that students who remain in the program improve over time gives new credence to advocates who said it was unreasonable to judge a program based on only one or two years of data.

“The results obviously cast further doubt on proponents’ claims that awarding vouchers to low-income students will immediately boost their math and reading achievement, but they also indicate that the negative initial effects on test scores seen in Louisiana, Ohio, and now Indiana are less concerning than it might appear,” said Marty West, a professor at Harvard, who reviewed the paper at Chalkbeat’s request.

The authors of the study suggest that private schools have gotten better as they have acclimated to new students who were more disadvantaged than those they previously served.

“Over time, voucher students may adjust to their new schools, and private schools may make adjustments that better meet the educational needs of voucher students,” the authors write, though they note that their research can’t confirm either hypothesis.

Research also released Monday on year three of Louisiana’s voucher program showed that negative results in early years of the program dissipated for some students in some subjects, but found continued negative effects for those in younger grades.

“What’s interesting about the Indiana results is that they’re consistent with the Louisiana results … in that they start out pretty negative and get less negative over time,” said Matt Chingos of the Urban Institute who has studied private school scholarships in New York City.

West noted the large achievement drops for those leaving the program may not be a bad sign.

“The fact that the students who switch back were disproportionately those who saw big drops in achievement is encouraging,” he said. “It does suggest that any large negative effects of voucher programs on achievement could be to some extent self-correcting.”

Doug Harris, an economist at Tulane University who has been critical of DeVos, said the the new research in Indiana “still has to give pause to anyone pushing broad federal or statewide [voucher] programs.”

“There are still no examples of statewide programs producing overall positive academic effects,” he said.

Other researchers praise study, but point to limitations

The results come with several important caveats.

First, because vouchers were not assigned through random lottery — unlike in some state programs, like Louisiana’s — the researchers can’t be confident that the results only capture the impact of receiving a voucher, a point the study acknowledges.

“Choosing to apply for and receiving a voucher depends on the active choices of parents and their children,” Berends and Waddington write.

Other researchers said the Indiana study does a good job controlling for that selection bias, though.

“The study is well done,” said Harris. “They try many different methods and the results hold up well.” He did note that there was some evidence that low-income students who took a voucher were more advantaged than poor students in public schools, suggesting the possibility of an “upward bias” in the results.

Second, the Indiana research is only able to look at a subset of the thousands of students who have used a voucher in Indiana to date — late elementary and middle school, low-income students who switched from public to private school. Similarly, the researchers only had data on a small number of students remained in the program for four years. That’s a limitation in using the study to draw conclusions about other students, West said.

Enlow and West both noted that the study only measures academic success with state test scores.

That, West said, means that it “can’t speak to how voucher use may have affected other student outcomes or families’ satisfaction with their child’s school.”