schools of the future

As ed reformers urge a ‘big bet’ on personalized learning, research points to potential rewards — and risks

Philanthropists and school leaders need to make a “big bet” on dramatically reshaping schools, according to the leaders behind last week’s major education conference.

Re-imagining learning, schools of tomorrow, personalized learning, jobs of the future — these were the watchwords at the New Schools Venture Fund Summit. Ideas about how to reinvent schooling were more prominent than even hot-button topics like school integration or vouchers.

“The world has changed dramatically … and our schools have struggled to keep up,” said New Schools CEO Stacey Childress at the summit’s opening session. “We just think it’s time to update the way schools work so they better prepare students for success in today’s world.”

The solution, according to some, is a focus on innovative school models — particularly ones that use technology to “personalize” teaching based on students’ needs, desires, strengths, and weaknesses.

The gathering was underwritten by deep-pocketed funders known for backing technology-based education initiatives, including the Gates Foundation and the relatively new Chan Zuckerberg Initiative. (Gates is a supporter of Chalkbeat, as are the Walton Family Foundation and the Carnegie Corporation, which were also major sponsors of the event.)

Advocates are extremely optimistic about investing in this new breed of schools. A report released by New Schools at the end of last year made an aggressive call to direct $4 billion in philanthropy toward creating, studying, and promoting this sort of innovation. Doing so, the authors write, will produce an estimated 200 to 500 percent return on investment.

It’s a remarkable claim, but perhaps an overly rosy one: the author of the study that New Schools relied on to make such a strong assertion says it’s likely inflated. And although some studies point to the benefits of specific technology programs, the research about whole schools using these approaches remains in its infancy.

As the ideas gain prominence, skeptics fear that the push amounts to the latest fad in education.

“What I see … is people imagining that if we just design the school with new models we will be able to satisfy the needs of the future,” said Ben Riley, head of the group Deans for Impact and former New Schools staff member. “The graveyard of people thinking they could successfully predict the future and then finding out that they were wrong about that has a lot of tombstones.”

The appeal of innovative schools and a personalized approach

Amidst a sea of buzzwords, it can be hard to define school innovation or personalized learning. (Some even argue that trying to pin down a clear meaning is misguided.)

Roughly speaking, though, the idea epitomized at the New Schools summit focuses on expanding technology in schools to better tailor teaching to specific students’ needs and desires.

The New Schools report points to several traits of these schools: maximizing “time, pace, instructional methods and outside experiences”; using “an expanded definition of student success”; ensuring students “feel ownership of their learning”; more frequent use of technology; and ensuring students build trusting relationships.

Advocates often point to Summit Public Schools, a charter school network in California that embraces a technology-infused model. A favorable case study authored by David Osborne of the reform-oriented Progressive Policy Institute calls them “schools of the future.”

Students at Summit spend about 16 hours a week — half of their school time — learning via computer. As Osborne describes it: “Teachers were there to answer questions, make suggestions when kids got stuck, and check their progress, but students were in charge of their own learning. They worked at their own pace, and when they felt they had mastered a concept, they took a 10-question assessment. If they could answer eight of the questions correctly, they checked that off and moved on to the next topic.”

Supporters of that approach highlight an analysis released by the RAND Corporation and the Gates Foundation. The study, which looked at schools funded by Gates because of their “promising approaches to personalized learning,” found that students made much larger gains on standardized tests compared to peers with similar characteristics attending different schools.

There is broader evidence that specific technology programs can improve achievement. Studies of computer programs designed to help students with algebra and early literacy, as well as the math program used by the Rocketship charter network, have shown benefits for students.

“The findings in general are very positive, especially for math,” said Andre Nickow, a Northwestern graduate student who worked on an analysis of technology-based personalized learning that was recently presented at a research conference.

Proponents also say the push for personalized learning is based on a deeper understanding of how kids learn — and how they can benefit from individualized instruction.

“The evidence base for the benefits of 1:1 mastery-based instruction is quite strong,” said Debbie Veney, a spokesperson for New Schools, pointing to a 1984 analysis of individual tutoring by prominent educational psychologist Benjamin Bloom, creator of the well-known Bloom’s taxonomy.

But providing each student a personal tutor is prohibitively expensive; the idea is to use technology to provide that personalization at a fraction of the cost.

“The innovative schools we and others support are working to find ways to produce similar academic results for students by using a mix of instructional approaches,” Veney said. “The early results of schools working on this challenge are quite promising, including the schools in the RAND study.”

Students at University Prep, a Denver elementary charter school, work on a computer-based assignment .
PHOTO: Marc Piscotty

The concerns about existing research

Indeed, that RAND study of innovative schools using personalized learning is often cited in these conversations. But there are a few reasons to view the findings cautiously.

The first is how New Schools frames the results in its report. Based on the RAND analysis, New Schools estimates a 200–500 percent return on the $4 billion the group argues philanthropists should invest over 10 years. But John Pane, author of the RAND study, disputes those projections.

“I was not really keen on some of the leaps that they were making based off of our data,” he told Chalkbeat. Pane himself estimated a potential return on investment and said he found “vastly smaller numbers” than New Schools did. (He declined to share his exact figures because his results have not gone through peer review yet.)

Veney said that the method in the report — converting effects on test scores into days of learning — is common in education research, including the CREDO charter studies.

The report has another major limitation, which Pane acknowledges: the schools in the study had to go through a rigorous application process in order to receive funding. That means the academic gains described in the study may not be the result of the schools’ use of personalized learning per se, but simply the possibility that only high-performing schools were examined.

“For me, the headline was good schools do good things,” said Alex Hernandez, a partner at the Charter School Growth Fund, which invests in new school models.

Regarding the two leading “pioneers” identified by New Schools, the research is also limited. Studies of the New Classrooms model, a personalized learning program that began in New York City known as Teach to One, have found mixed results. There does not appear to be any external empirical research on Summit Public Schools, though an internal analysis says students in the schools make large gains on standardized tests.

Skeptics say that the theory behind personalized learning has some inherent flaws, too.

For instance, Diane Tavenner, Summit’s CEO, previously said, “Look at the economy: it’s not about concrete knowledge, it’s about higher-order thinking skills, and the ability to perpetually learn and grow.”

Sentiments like that worry Ben Riley, who argues that the best way to train students for the jobs of tomorrow is to “provide them with a really rich, broad comprehensive education that gives them the knowledge base that allows them to adapt to those new jobs.”

“If there’s one thing that cognitive science has shown over and over again, it’s that our ability to understand new ideas depends on what ideas we already know,” said Riley, whose group released a report compiling research on the science of learning.

Meanwhile, unmentioned in New Schools’ report is research on fully virtual schools, in which instruction is delivered exclusively online. Although supporters might not categorize this approach as personalized learning, the schools are certainly different from traditional schools in their reliance on technology. And yet the results to date have been abysmal — students have generally seen large drops in test scores relative to those in brick-and-mortar schools. This suggests that expanding technology in education can come with significant risks.

If it fails, move on — but who is left behind?

New Schools acknowledges that these innovative schools aren’t a sure thing. (It is calling the plan a “big bet,” after all.) But trying out new structures for schools is necessary to prepare more students to succeed in college, especially the black, Hispanic, and low-income students who haven’t been served well by typical schools, they argue.

“If after several years this approach isn’t living up to the potential we imagine, let’s change course,” the report states. “But let’s also evaluate every other idea for how to direct the $20 billion in education philanthropy over the next 10 years based on concrete estimates of the improved outcomes we should expect for students.”

As schools experiment, “We will fail,” said Derwin Sisnett, cofounder of the Gestalt Community Schools charter network, at the summit.

But critics of the approach say such failure could lead to collateral damage.

“Should philanthropists scrap the initiative after five years, 1,700 schools will be left to deal with the aftermath and lost funding despite the many constraints they face,” wrote Jeffrey Snyder of Cleveland State University in a critical review of the New Schools report for the National Education Policy Center, which is partially funded by teachers unions.

Veney of New Schools said the group’s vision is to use philanthropic funds temporarily to jumpstart innovation. “Our proposal is to support educators who want to make the shifts with the resources they need to do it well,” she said, “and to ensure that the designs and practices they are adopting are sustainable with public funding after the first few years.”

Snyder, though, writes that the risks extend beyond the financial considerations. “Especially concerning should be the potential for this philanthropic shift to continue (and exacerbate) reform churn — the process where schools move quickly among reforms, never allowing any to take root.”

Fact check

Why the school spending graph Betsy DeVos is sharing doesn’t mean what she says it does

PHOTO: U.S. Department of Education

U.S. Secretary of Education Betsy DeVos’s message at an event on Thursday was clear: education leaders have been throwing good money after bad policy.

To make the point, she shared a graph that appears to show that school spending has risen dramatically over the last three decades as student outcomes barely budged.

“Scores [are] continuing to muddle along, unremarkable,” she told one of her predecessors, Bill Bennett, at a Reagan Institute event. “And yet look at the spending. This is not something we’re going to spend our way out of.”

This visualization, which DeVos also shared on Twitter, has become a staple in some education policy circles. Even Bill Gates has offered a version of it. But the graph is misleading, and several recent studies have in fact shown that students gain — in higher test scores, graduation rates, and earnings as adults — when they attend schools that spend more money. And while DeVos highlighted scores in reading, gains in math have been fairly substantial over part of that period.

A spokesperson for DeVos did not immediately respond to a request for additional comment.

Here’s why it doesn’t really mean what DeVos says it does.

The biggest problem with the graphic is that it conflates correlation and causation.

Simply pointing to two trends that coincide doesn’t say anything about how one affects the other. It’s possible that if education spending hadn’t increased, test scores would have actually declined. Maybe the extra money actually stopped a dramatic deterioration in school quality, for example. There’s just no way to know based on this data.

Here’s another example: Over the past couple decades, charter schools and vouchers have expanded dramatically. By DeVos’s own logic, those would be failed reforms because they didn’t coincide with a sufficient increase in NAEP scores. But that would also be an unfair conclusion, since we don’t know what would have happened without the increase in choice programs.

The two trends are also put side-by-side in an arbitrary way.

This graph has two y axes: one based on school spending, and the other based on scale scores on the National Assessment of Educational Progress. But these two variables are very different. Should an extra $1,000 correspond with 1 point on NAEP scores? Five points?

The decision is subjective, but it determines how large the spending changes look and how small the NAEP changes look, too.

Spending is up in education, but it’s up elsewhere too.

It’s true that in inflation-adjusted dollars, school spending has risen. The economy has also grown over that same time. That matters, because if spending were to hold steady in education while increasing in other areas of the economy, it might become increasingly hard to recruit and retain teachers of equal quality.

Indeed, while education spending has risen, teacher pay compared to similar professionals has fallen. (At the same time, the number of teachers and the number of non-teaching staff in schools has increased significantly, which some have argued illustrate bad spending priorities.)

The story would also look different for math scores.

NAEP scores have been relatively flat for the past decade, but in math they increased substantially the decade beforehand. In fact, a graph could be made showing a tight correspondence between math results and spending.

That’s exactly what Northwestern University economist Kirabo Jackson did. His approach was similar to DeVos’s, with a few changes. He compared NAEP scores in math to the cumulative spending on the average student in the years leading up to the test — the idea being that scores reflect not just the last year of schooling, but several years before that.

Source: Education Next

His graphs look quite different than DeVos’s, simply by tightening the range of scores, looking at math rather than reading, and tweaking the spending measure. (Jackson said that a similar pattern exists for reading, but that it’s less pronounced.)

Recent research has generally shown that school spending really does matter.

To be clear, Jackson’s graphs are also correlational, and like DeVos’s cannot prove cause and effect.

That’s why he and others have conducted sophisticated statistical analyses to isolate the impact of resources on schools. In one recent study, Jackson and colleagues compared states that made more spending cuts in the wake of Great Recession, and found that they did worse on NAEP as a result. In other words, there is a decent case that the recent stagnation in NAEP scores is due at least in part to spending cuts.

That jibes with a larger body of research. Other recent national studies have linked more spending to better NAEP scores, higher graduation rates, better economic outcomes, and greater social mobility. Studies in specific states including California, MassachusettsOhio, and New York have also found test score gains from spending increases.

Of course, none of that proves that the additional money spent over the past few decades have been well used. It doesn’t even prove that those extra dollars have been helpful — the above studies are of specific places or time periods, and their individual approaches could be disputed, too.

There is one recent study, which DeVos often cites, showing that a federal turnaround program that involved a substantial infusion of money for struggling schools didn’t make a noticeable difference.

Still, the graph that Devos offers is not, on its own, a strong case that resources for schools don’t make a difference. Available evidence suggests the opposite.

“If reduced spending contributed to the problem, it seems reasonable to conclude that increased spending may alleviate it,” said Jackson, the Northwestern researcher. “This is not to say that increased spending is some silver bullet, but rather that it should probably be part of a comprehensive set of policies to improve U.S schools.”

after douglas

Betsy DeVos avoids questions on discrimination as school safety debates reach Congress

Education Secretary Betsy DeVos prepares to testify at a House Appropriations Labor, Health and Human Services, Education and Related Agencies Subcommittee hearing in Rayburn Building on the department's FY2019 budget on March 20, 2018. (Photo By Tom Williams/CQ Roll Call)

U.S. Education Secretary Betsy DeVos fielded some hostile questions on school safety and racial discrimination as she defended the Trump administration’s budget proposal in a House committee hearing on Tuesday.

The tone for the hearing was set early by ranking Democrat Rep. Rosa DeLauro, who called aspects DeVos’s prepared remarks “misleading and cynical” before the secretary had spoken. Even the Republican subcommittee chair, Rep. Tom Cole, expressed some skepticism, saying he was “concerned about the administration continuing to request cuts that Congress has rejected.”

During nearly two hours of questioning, DeVos stuck to familiar talking points and largely side-stepped the tougher queries from Democrats, even as many interrupted her.

For instance, when Rep. Barbara Lee, a Democrat from California, complained about proposed spending cuts and asked, “Isn’t it your job to ensure that schools aren’t executing harsher punishments for the same behavior because [students] are black or brown?” DeVos responded by saying that students of color would benefit from expanded school choice programs.

Lee responded: “You still haven’t talked about the issue in public schools as it relates to black and brown students and the high disparity rates as it relates to suspensions and expulsions. Is race a factor? Do you believe that or not?” (Recent research in Louisiana found that black students receive longer suspensions than white students involved in the same fights, though the difference was very small.)

Again, DeVos did not reply directly.

“There is no place for discrimination and there is no tolerance for discrimination, and we will continue to uphold that,” she said. “I’m very proud of the record of the Office of Civil Rights in continuing to address issues that arise to that level.”

Lee responded that the administration has proposed cuts to that office; DeVos said the reduction was modest — less than 1 percent — and that “they are able to do more with less.”

The specific policy decision that DeVos faces is the future of a directive issued in 2014 by the Obama administration designed to push school districts to reduce racial disparities in suspensions and expulsions. Conservatives and some teachers have pushed DeVos to rescind this guidance, while civil rights groups have said it is crucial for ensuring black and Hispanic students are not discriminated against.

That was a focus of another hearing in the House on Tuesday precipitated by the shooting last month at Marjory Stoneman Douglas High School in Parkland, Florida.

Rep. Jim Sensenbrenner, a Wisconsin Republican, falsely claimed in his opening statement that Broward County Public Schools rewrote its discipline policy based on the federal guidance — an idea that has percolated through conservative media for weeks and been promoted by other lawmakers, including Florida Sen. Marco Rubio and Utah Sen. Mike Lee. In fact, the Broward County rules were put into place in 2013, before the Obama administration guidance was issued.

The Manhattan Institute’s Max Eden, a leading critic of Obama administration’s guidance, acknowledged in his own testimony that the Broward policy predated these rules. But he suggested that policies like Broward’s and the Obama administration’s guidance have made schools less safe.

“Faced with pressure to get the numbers down, the easiest path is to simply not address, or to not record, troubling, even violent, behavior,” he said.

Kristen Harper, a director with research group Child Trends and a former Obama administration official, disagreed. “To put it simply, neither the purpose nor the letter of the federal school discipline guidance restrict the authority of school personnel to remove a child who is threatening student safety,” she said.

There is little, if any, specific evidence linking Broward County’s policies to how Stoneman Douglas shooter Nicholas Cruz was dealt with. There’s also limited evidence about whether reducing suspensions makes schools less safe.

Eden pointed to a study in Philadelphia showing that the city’s ban on suspensions coincided with a drop in test scores and attendance in some schools. But those results are difficult to interpret because the prohibition was not fully implemented in many schools. He also cited surveys of teachers expressing concerns about safety in the classroom including in Oklahoma CityFresno, California; and Buffalo, New York.

On the other hand, a recent study found that after Chicago modestly reduced suspensions for the most severe behaviors, student test scores and attendance jumped without any decline in how safe students felt.

DeVos is now set to consider the repeal of those policies on the Trump administration’s school safety committee, which she will chair.

On Tuesday, DeVos said the committee’s first meeting would take place “within the next few weeks.” Its members will be four Cabinet secretaries: DeVos herself, Attorney General Jeff Sessions, Secretary of Health and Human Services Alex Azar, and Secretary of Homeland Security Kirstjen Nielsen.

Correction: An earlier version of this story said that Rep. Barbara Lee was from Texas; in fact, she represents a California congressional district.