Bumpy runway

Emails reveal months of missteps leading up to Tennessee’s disastrous online testing debut

PHOTO: Chalkbeat Photo Illustration

Tennessee education officials allowed students and teachers to go ahead with a new online testing system that had failed repeatedly in classrooms across the state, according to emails obtained by Chalkbeat.

After local districts spent millions of dollars on new computers, iPads, and upgraded internet service, teachers and students practiced for months taking the tests using MIST, an online testing system run by North Carolina-based test maker Measurement Inc.

They encountered myriad problems: Sometimes, the test questions took three minutes each to load, or wouldn’t load at all. At other times, the test wouldn’t work on iPads. And in some cases, the system even saved the wrong answers.

When students in McMinnville, a town southeast of Nashville, logged on to take their practice tests, they found some questions already filled in — incorrectly — and that they couldn’t change the answers. The unsettling implication: Even if students could take the exam, the scores would not reflect their skills.

“That is a HUGE issue to me,” Warren County High School assistant principal Penny Shockley wrote to Measurement Inc.

Tennessee Education Commissioner Candice McQueen speaks with reporters in February about technical problems with the state's new online assessment.
PHOTO: Grace Tatter
Tennessee Education Commissioner Candice McQueen speaks with reporters in February about technical problems with the state’s new online assessment.

The emails contain numerous alarming reports about practice tests gone awry. They also show that miscommunication between officials with the Tennessee Department of Education and Measurement Inc. made it difficult to fix problems in time for launch.

And they suggest that even as problems continued to emerge as the test date neared, state officials either failed to understand or downplayed the widespread nature of the problems to schools. As a result, district leaders who could have chosen to have students take the test on paper instead moved forward with the online system.

The messages span from October until Feb. 10, two days after the online test’s debut and cancellation hours later. Together, they offer a peek into how Tennessee wound up with a worst-case scenario: countless hours wasted by teachers and students preparing for tests that could not be taken.

October: ‘Frustration … is definitely peaking’

Leaders with the Education Department, local districts and Measurement Inc. all knew that Tennessee’s transition to online tests wouldn’t be easy. So the test maker and the department developed a plan to identify weaknesses: stress tests they called “Break MIST” to tax and troubleshoot the online system.

They all had a lot riding on a smooth rollout. Tennessee was counting on the scores to assess whether students are measuring up to new and more challenging standards, to evaluate teachers, and to decide which schools to close. Districts, even the most cash-strapped, had invested millions of dollars on new technology. And Measurement Inc., a small company headquartered in Durham, was looking to prove that it belonged in the multibillion-dollar testing industry’s top tier.

The first “Break MIST” day on Oct. 1 was a mess — as expected. Students in the eastern part of the state logged on without issue, but the system stumbled as the majority of students started their tests an hour later.

That morning, emails show that Measurement Inc. received 105 calls reporting problems. The company noted particular problems in districts using iPads. Officials from the testing company assured the state that the bugs could be fixed, and the education department passed the message on to the public.

Department officials said nearly 1.5 million practice tests were completed successfully over the course of the fall. But emails show that even on days that weren’t meant to tax the system, problems emerged.

On Oct. 20, students in some districts were taking practice tests when “everything quit,” according to a state official who summarized complaints that local technology coordinators were swapping by email.

“Not very reassuring,” wrote Randy Damewood, the IT coordinator in Coffee County.

“Not good news,” agreed John Payne, director of technology for Kingsport City Schools, who suggested that his own district’s tests were working that day.

“The frustration among teachers and central office staff is definitely peaking,” wrote Eric Brown, a state official.

But there was more frustration to come, much of it behind the scenes at the Education Department.

December to January: Communication falters

Even after Measurement Inc. and department officials worked together to address problems during practice tests, the department still wasn’t confident in the online system. They weren’t sure whether problems were due to local infrastructure or something bigger. Officials planned two more “Break MIST” days in January to find out.

But they didn’t involve Measurement Inc. in the planning, at least according to company officials who wrote to the department to say they learned of those plans only after being copied on an email sent to local superintendents by Education Commissioner Candice McQueen.

That message was one of many in which officials with the state or the testing company expressed frustration about communication in the weeks leading up to the testing period.

One tense exchange dealt with the problems faced by students taking practice tests on iPads. “Will the iPad platform be ready for primetime in the spring?” Assistant Commissioner Nakia Towns asked Measurement Inc. officials on Dec. 3. “I feel like we need to be honest on this one.”

The test maker did not email a response, and Towns raised the issue again a month later and indicated that she was still waiting for an answer. “I had asked the question very directly in December,” she wrote Measurement Inc. on Jan. 6. “We urgently need an update.”

It took five more days, until Jan. 11, for her to get an answer. A reply from a Measurement Inc. testing expert blamed the problem on Apple but suggested the company had a “workaround.”

The next day, 504 students in Dyer County, about 80 miles north of Memphis, attempted to take the exam, many of them using iPads. Not one was able to complete the test because questions took too long to load, according to a report from Measurement Inc.’s call center. (Another half-million tests were completed successfully during January, according to department officials.)

Henry Scherich
Henry Scherich

In an interview this week, McQueen told Chalkbeat that Measurement Inc. never fixed the iPad problem and that state officials called Apple themselves looking for a solution. She was still looking for an answer on Jan. 21, when she tried to speak directly with Measurement Inc. President Harry Scherich.

“She is wondering if there is any way for you to find even 15 minutes today for a call,” McQueen’s chief of staff wrote. “Commissioner will make herself available. We need to speak to someone who would be able to make a decision concerning technology in an effort to get communication to directors of schools today.”

Scherich, who was in Michigan meeting with that state’s education department, initially said he did not have time to speak with McQueen. (Measurement Inc. is one of two companies producing Michigan’s new exam.) Later that day, he agreed to speak.


McQueen said she and her team came to a conclusion the next day: The test wouldn’t work on iPads. They emailed and called districts that had purchased tablets for testing and recommended a switch to paper.

February: A last-minute warning gets too little attention

Even as tensions mounted and glitches piled up, both the department and Measurement Inc. projected confidence about what would happen on Feb. 8, when the test would go live for most Tennessee schools. State officials even invited reporters to Department of Education offices on Feb. 3 to say they were optimistic about the rollout.

But behind the scenes, they were preparing for the worst. McQueen asked the test maker’s call centers to prepare for a major outage, something a Measurement Inc. employee told her was “very unlikely.”

She also emailed districts telling them they should consider switching to paper tests if their students were waiting too long for questions to load. She gave them three days to decide.

Just 15 of Tennessee’s nearly 150 districts took her up on the offer, McQueen told Chalkbeat.

But emails show that the state knew that most districts were having difficulties. When one district’s technology coordinator asked the state for a list of districts ready for the online exam, officials came up short.

“I don’t think I can answer that with any confidence,” the department’s top technology officer wrote.

Five days later, on Monday, Feb. 8, the test officially began. Again, the system handled the first set of test takers but broke down when the rest of the state’s students logged on.

As students stopped being able to connect or saw their tests freeze, emails show that technology directors began frantically contacting each other.

“Has anyone else had MIST drop out on them?” the director from Houston County Schools asked. A chorus of technology directors from other districts replied in the affirmative.

Within hours, Tennessee had ended its foray into online testing. First, McQueen told districts to suspend the exams, then directed them to give up on the online platform altogether.

“We are not confident in the system’s ability to perform consistently,” she wrote in an email to school superintendents that afternoon.

McQueen told Chalkbeat that officials started the day “in good faith,” with an assumption that Measurement Inc. had resolved problems adequately. Scherich told Chalkbeat that he’s still unconvinced that the problems were the company’s fault. He suggested that Tennessee’s decision to cancel testing came too soon.

Either way, the department’s top technology official put it simply when he emailed McQueen on the day of the failure. “It appears that greater procedural and operational rigor could have prevented the network outage,” Cliff Lloyd wrote to McQueen.

The debacle was just what Ravi Gupta, the CEO of a Nashville-based charter school, was worried about when he pressed the state in January for more transparency about the status of the online platform.

“It would be a betrayal of our students’ hard work if adult technical failures stood in the way of their success,” Gupta wrote to McQueen.

In the end, that’s exactly what happened.

Clarification (June 28, 2016): This story has also been revised to clarify the impact of the department’s communications on district testing decisions. It has also been updated to include new information about successful practice tests.

History alive

Inspired by Hamilton, Colorado students perform their own raps and poems on the big stage

PHOTO: Photo by Hyoung Chang/The Denver Post
From left, West Leadership Academy's Alexandra Andazola Chavez, Jose Torres Andazola, Rossy Martinez Sanchez, and Zehydi Chaparro Rojas perform "The Story of Peggy."

The plush red seats at the Wednesday matinee of Hamilton in Denver were filled with 2,700 teenagers who’d spent weeks studying a special curriculum about the hip-hop musical’s namesake, Alexander Hamilton, and the other Founding Fathers. Even though the show’s four-week Denver run had been sold out for months, the teenagers were seeing it for free.

Some of them had dressed for the occasion in high-heeled boots and three-piece suits. Others wore jeans and Converse. They represented 38 Colorado high schools that serve high proportions of students from low-income families, and many of them were students of color.

That’s notable because most of the cast of Hamilton are actors of color. Hamilton, George Washington, and Thomas Jefferson are played by black and Latino actors, a decision creator Lin-Manuel Miranda has said reflects America’s racial makeup and is meant to pull the audience into the story of an immigrant, Hamilton, who played an important role in the nation’s founding.

Before the show, 23 students took the stage to perform their own spoken word poems, raps, monologues, and scenes inspired by what they’d learned from the Hamilton Education Program curriculum, which was devised in part by Miranda and has its own hashtag: #EduHam.

“My body felt electrified,” said Josiah Blackbear, a 15-year-old sophomore at West Early College in Denver, who performed a rap he’d written about Alexander Hamilton. “The words I was speaking brought power and truth to the rest of the venue.”

Here is video of six of the student performances, including one entirely in Spanish.


During Memphis visit, former Newark schools chief touts ways to change student discipline

PHOTO: Governor's Office/Tim Larsen
Cami Anderson when she was superintendent of schools in Newark, New Jersey.

As the top schools chief in Newark, Cami Anderson was horrified at the strict discipline policy she saw in one of her high schools. Since then, she has left the New Jersey district and taken her ideas on the road about reducing suspensions and moving away from exclusionary discipline practices.

This week, Anderson came to Memphis as part of her Discipline Revolution Project at the invitation of Stand for Children Tennessee, The Community Foundation of Greater Memphis, School Seed, and Shelby County Schools. The New Teacher Project is partnering with her on the national tour.

Anderson has been meeting with Shelby County Schools administrators and board members as well as charter school leaders, philanthropists, education advocates, and students. Her time will culminate in a public event hosted by Stand for Children on Thursday at the National Civil Rights Museum.

Chalkbeat sat down with Anderson after she explained to a group of about 40 charter leaders her six focus areas to reduce classroom disruptions while also preventing sending students home when they’re in trouble. (This interview was edited for clarity and brevity.)

Related story: Tennessee students more likely to be suspended if they’re black boys — or live in Memphis

Question: How did you land on student discipline as an area you wanted to focus on?

Answer: If there’s actually a thread in my career, it’s this. I essentially ran the system of supports for the kids in New York City who are on their last stop on the train, so to speak. I’ve always worked with kids who are marginalized, the ones who really struggled in school. So, I’ve spent a lot of time thinking about what we need to get better at collectively to serve all kids, to really embrace the “all means all.” That’s been my lifelong question.

The three areas to me where inequities are most obvious are: enrollment policies, how we handle discipline, and mobility and how a kid stays connected to school. Discipline is where it comes to a head. It’s both a place where our collective inability to reach all kids shows up and it’s also an opportunity if we actually figure out how to prevent young people from misstepping in the first place, but then respond in healthy ways when they do Then we’d actually start to solve the broader equity issues.

Q. School leaders say they don’t want to have a lot of suspensions because students miss out on class. But they’re also not sure what to replace suspensions with to manage student behavior well. What would you say to them?

"You can’t just be against something. You have to be for something."Cami Anderson

A. That’s one of the main reasons we started Discipline Revolution Project. We don’t want you to do X, whatever X is: suspend kids, use corporal punishment. But educators are saying, rightfully so, then what are we doing? Our whole framework is trying to answer that question and give them tools to get to the “why” behind finding alternative responses.

Most people who use punitive or exclusionary discipline don’t actually think it works that well. They just don’t have a lot of other tools. So, when you give folks a lot of other tools and they find that it works, it’s a very powerful thing. When people try out a restorative conference, they say “Oh, I feel better. The kid feels better. And we actually got back to the lesson faster.” You can’t just be against something. You have to be for something.

I’ve heard a lot of demand for basics of restorative practices (conflict resolution between students and students and teachers), though I don’t think they should stop there. They want training for student support teams. And overwhelmingly, the places I’ve been want to talk about how teacher bias plays into who gets disciplined, but they don’t know how to start the conversation and for it to be productive.

Q. Memphis’ two school districts have emphasized a bottom-up approach on discipline reform: adding behavior specialists, school counselors, soliciting support from principals and teachers. How have you seen other districts do it?

"Policy-level change is critical but insufficient if you only do that."Cami Anderson

A. I’ve seen districts lead with policy and only make statements declaring they will cut suspensions in half or put a moratorium on suspensions or rewrite their policy. Policy-level change is critical but insufficient if you only do that. What you see is folks who are actually on the ground working with students may not have the strategies to replace it with something productive. That causes people to be more entrenched in their views that discipline reform wouldn’t work, some schools subtly pushing kids out, underreporting discipline data, all that.

I’ve also seen the opposite where it’s all about professional development and capacity but at no point is there is any accountability for those same schools, for example, that suspend 90 percent of the kids. People watch what you do, not what you say. If you don’t align your policies and your actions with your values, then you also have limits to the impacts you have for kids.

Frustrated with high suspension rates, Memphis schools shift to restorative justice

I’ve come to believe you need all of it and you need everyone working together. Stop admiring the problem and get on to the solutions.

Q. What pushed student discipline practices more widely into the national conversation? What have you observed from the conversation here in Memphis?

A. People are looking at data, which is a good thing, and seeing patterns like everyone else. Another thing is I believe a lot of people who got into education reform are completely dedicated to equity. And now they’re seeing this side of it, and like someone said in the training today, they feel a sense of “healthy guilt.” I think it’s great they’re having the courage to be honest. And then a lot of folks had kids. You start thinking, “Do I want any of that happening to my own kid?” I’m personally heartened and encouraged and motivated to see a collective sense of responsibility and focus on this.

There’s a lot of energy and candor in Memphis about this issue. Some other cities I’ve been in think they have it figured out when they don’t. When there’s that much energy, I think anywhere — including in Memphis — people can be tempted to devolve into the blame game, no matter what district or charter hat you wear. That energy can be the greatest asset or greatest liability.

Study: When Chicago cut down on suspensions, students saw test scores and attendance rise

Q. The school shooting in Parkland has been a catalyst for more conversations about the trauma students bring into the classroom — conversations that were already happening about violence in low-income communities of color. What would you say to school leaders on how to address that?

A. I’m most interested to know what adults can do to mitigate those risk factors for young people who experience trauma. I feel like it could take us down a very bad path to just observe that there are things called “adverse childhood experiences.” To me, that’s not enough. The question then is what are the environments and strategies that we can put in place as educators and adults to mitigate the impact of those traumatic experiences. Things like relationships, trust, consistency, high expectations, high supports, and support healthy identity development especially in times of conflict. We know from research that young people who face long odds who ultimately prevail, they are exposed to environments that really embody those things.

You can both be aware of and acknowledge those experiences that make it harder for them to succeed in school. But if you stop there, I don’t think you’re doing justice to young people. There are things we can do in schools to help create the environment to help them succeed.