Another pause

It’s official! Results from Tennessee’s ugly testing year won’t count for much of anything

PHOTO: Marta W. Aldrich
Gov. Bill Haslam and his education chief, Candice McQueen, speak with reporters Monday about how Tennessee will handle standardized test results this year because of technical problems administering the exams by computer.

Tennessee teachers and school districts can decide how to use this year’s standardized test results and won’t be penalized for low growth scores after another year of problems with the state’s computerized exam.

The state also will shift some responsibilities to a different testing company while deciding whether to extend Minnesota-based Questar’s contract to give the test past November, Education Commissioner Candice McQueen announced on Monday.

In addition, the state has hired two outside groups to sort through all that went wrong with TNReady during almost a month of testing that ended last week. One will look at the validity of the results to see if frequent online interruptions made the scores unreliable. The other will scrutinize Questar’s technology systems to determine why so much went awry.

The actions were announced as McQueen and Gov. Bill Haslam faced reporters together for the first time in the wake of this year’s sloppy return to statewide computerized testing.

After weeks of being on the defensive, Haslam’s administration sought to take control of the situation and emphasize that testing — done correctly — is critical to improving student achievement across the state.

I still have full confidence that testing is the right thing to do,” said the Republican governor, finishing the last year of an eight-year term. I’m frustrated like everybody else that we had issues with the online portion of this. But having said that, do I think the test is a good test? I do.”

Haslam also said Tennessee must forge ahead with computerized testing.

We’re one of only 10 states that has not already moved [completely] to online testing. And so it’s not just that’s where the world is going; that’s where the world is. And our students have to be prepared,” he said.

The decision to shield students, teachers, and schools from accountability for poor growth scores falls in line with emergency legislation passed by state lawmakers last month as reports of TNReady’s technical problems escalated. After weeks of studying the two new laws, McQueen and her team offered their first analysis of what the legislation means:

Teacher evaluations. The state still plans to include student growth scores in evaluations, but each teacher will have “complete control to nullify” that portion if they choose to rely solely on other measures, McQueen said.

Student grades. Local school boards will decide whether to incorporate TNReady scores into this year’s final grades. Many districts already have begun that process, and most are opting to exclude the results this year.

School ratings. Tennessee’s A-F rating system will not launch this fall as scheduled, although the state still will publish the achievement results that would have gone into them.

Priority schools. As planned, the state will release its “priority list” this fall of the 5 percent of lowest-performing schools, but this year’s test results will not be a factor. Instead, the list will be based on two years of previous scores for high schools and one year for lower-grade schools. “We will not be moving any schools based on that data into the Achievement School District,” McQueen said of the state-run turnaround program that takes over local schools and assigns them to charter operators.

Whether the adjustments put Tennesseee out of compliance with federal law remains to be seen, though. The 2015 Every Student Succeeds Act requires that student achievement — as measured by tests like TNReady — be part of each state’s plan for holding struggling schools accountable.

“We’ve been in lots of conversations with the U.S. Department of Education, and they are continuing to work with us on that,” McQueen acknowledged. “So what we will do is create something called a comprehensive support list, which is required under the Every Student Succeeds Act. That will include this year’s data, but there will not be any adverse action taken based on that comprehensive school list.

"It's important for all of us that we get this right. "Gov. Bill Haslam

After three years of testing problems, McQueen announced that Tennessee will create a “TNReady Ambassadors” program to improve customer service and will hire a full-time overseer to work with testing coordinators at the district level.

We did not have our expectations met in terms of customer service from Questar,” she said.

To that end, the state is reviewing its annual $30 million, two-year contract with Questar that expires on Nov. 30.

Haslam said changing companies in the middle of a school year wouldn’t be seamless because Questar will begin testing high school students on non-traditional block schedules this fall. There’s a little bit of a practical problem switching vendors right in the middle of that, so it’s part of negotiations we’re in the middle of,” he said.

McQueen praised Educational Testing Services, the New Jersey-based company that will take over the test’s design work while Questar focuses on test delivery. Also known as ETS, the vendor has had contracts with Tennessee since 2015 to create the state’s social studies and science tests, and to design many of its teacher certification exams. (ETS also owns Questar. Read the details here.)

It is a vendor that is well-known. It has a reputation for very high-quality work in terms of how they design tests,” she said.

Deeper dive: By getting testing wrong again, will Tennessee undo what it may be getting right?

Regrouping

After another bumpy testing year, Tennessee likely will slow its switch to online exams

PHOTO: Marta W. Aldrich
Members of Tennessee's testing task force listen to a presentation by Mary Batilwalla, deputy commissioner over assessment for Tennessee's Department of Education. The group offered feedback on options for transitioning to online testing after more problems occurred this year.

Tennessee education leaders are rethinking their timeline for adopting computerized testing after a parade of technical problems bedeviled students taking the state’s TNReady exam for a third straight year.

Most students are scheduled to test online next school year under a three-year transition plan. But since keyboard testing had significant challenges this year with half that number of students, Education Commissioner Candice McQueen is backing off from that timetable.

And while there’s disagreement over exactly how to move ahead, there’s consensus about one thing.

“We have a credibility issue,” said state Rep. John Forgety, “and we need to get it right one time.”

McQueen floated three options for the 2018-19 school year to members of her testing task force during its Wednesday meeting in Nashville:

  •     Returning to paper testing across all grades for one year;
  •     Computer testing for high school students; paper testing for grades 3-8;
  •     Computer testing for grade 6 through high school; paper testing for grades 3-5

Off the table, however, is the option that districts had this year to give computer tests to more grades than required by the state.

The state ordered that all high school students take the test by computer this year, but about 40 percent of districts also chose to go digital for at least some of their students in grades 5-8.

The early thinking had been that letting districts test more students than required would expedite Tennessee’s online switch if local leaders felt ready. But state officials now believe the piecemeal approach only complicated the process.

“We feel very strongly” about this decision, Deputy Education Commissioner Mary Batilwalla told the task force. “The complexity is really too great for us to overcome in ensuring that we have a seamless delivery.”

The 30-member task force of educators and advocates has been McQueen’s sounding board on TNReady and other testing issues, and she sought the group’s feedback one week after the state’s messy testing season ended.

“We don’t want to introduce any additional complexity. We want to eliminate complexity, eliminate risk,” said McQueen, who also is turning to superintendents and upcoming focus groups for advice about how to improve their TNReady experience.

PHOTO: Marta W. Aldrich
Education Commissioner Candice McQueen speaks at a 2017 event as Gov. Bill Haslam looks on.

McQueen will decide about digital vs. paper — and for which grades — by late June. She is leaning toward keeping high schools online and putting all lower grades on paper tests, but it’s not a done deal, she told Chalkbeat.

“The feedback we’re getting is for more to go online than not, and that’s very meaningful to hear,” she said.

Her boss, Gov. Bill Haslam, has made it clear that Tennessee is committed to eventually adopting computerized testing.

“It’s not just that’s where the world is going; that’s where the world is,” Haslam said earlier in the week.

About 300,000 students took TNReady online this year — the most ever since a wholesale switch to computers failed in 2016 under Measurement Inc. McQueen fired that testing company, hired Questar as its successor, and unveiled a new game plan to gradually wade back in. That approach worked well last year for the 24 districts that did a trial run for high schools, although later scoring errors detracted from Questar’s debut.

This year marked the return to statewide online assessments, beginning with Tennessee’s oldest students. But challenges included a cyber attack and lousy internet service when a dump truck cut a main fiber optic cable — examples that demonstrate the risks of computerized testing.

There are benefits, too, however. Digital exams are quicker to score, offer more flexibility in the types of questions asked, and ultimately cost less. Returning to all paper testing would cost an extra $11 million in printing and shipping costs.

One big advantage of paper-and-pencil testing is a shorter testing period. Three weeks were allotted to TNReady this spring because schools had to rotate their students in and out of testing labs to use a limited number of computers. That requires a lot of coordination and time.

Task force members agreed that reverting to paper would be a step backward, especially with the state’s focus on the technical skills needed for college and careers and the significant investments made by school districts to prepare for online testing.

But they were adamant that Tennessee needs a win next time around to rebuild trust in a test that many consider broken.

“There has been a serious erosion in confidence in state testing, whether it’s online or on paper,” said Shawn Kimble, director of schools in Lauderdale County. “If we fail again, where does that leave us as a state?”

It's done

Tennessee’s messy testing season is finished, but how much the scores will matter is up in the air

PHOTO: Sol de Zuasnabar Brebbia/Getty Images
Problems with computerized testing in high schools and some middle grades consumed the headlines this testing season for Tennessee's standardized assessment called TNReady.

Tennessee students turned in their last standardized exams on Wednesday, capping more than three weeks of on-again, off-again computerized testing that has bedeviled school communities and called into question the future of the state’s testing system.

Since TNReady testing began on April 16, more than 2.5 million test sessions have been submitted online by some 317,000 students — more than ever before in Tennessee, according to the state Department of Education.

But the numbers make for an awkward superlative given the problem-plagued administration of this year’s exams. Technical glitches have been blamed on a range of issues including a cyber attack, a fiber optic cable severed by a dump truck in rural Tennessee, and a system error that caused 1,400 students to take the wrong assessment.

Bottom line: Students have struggled to consistently log on, stay on, and submit their tests. The frequent interruptions have required schools to be nimble with their schedules and spurred state lawmakers last month to pass emergency legislation that weakens how the scores will be used.

“While this was a rocky experience on certain days of the testing window, and we are empathetic to the frustrations that our students and teachers have felt, they have completed a key moment in this transition to online assessment,” spokeswoman Sara Gast said in a late-day statement to reporters.

The state reported that no school system stopped testing due to computer problems — and that every district’s online completion rate was at or near 100 percent. The numbers were enough, she said, to meet the required federal threshold for 95 percent of students to complete a state assessment.

Another 300,000 or so younger students finished their paper-and-pencil tests as of last week. However, it was computerized testing in high schools and some middle grades that consumed the headlines.

“We are continuing to learn from what we experienced [with] this administration, both on paper and online, and identifying ways to improve,” said Gast, pledging a thorough state analysis of results “to identify any impact the online interruptions may have had.”

The state also will examine irregularity reports, which Gast said districts were encouraged to submit “if they felt the student did not have a chance to demonstrate his or her knowledge of the standards due to the online issue.”

But how the results will be used is up in the air due to legislation that appears to gut the state’s accountability systems for students, teachers, and schools. TNReady scores were supposed to be incorporated into students’ final grades, teacher evaluations, a new grading system for schools, and a “priority list” that determines interventions for the state’s lowest-performing schools. But Tennessee lawmakers were clear that “no adverse action” may be taken based on this year’s results.

That may also put the state at odds with a federal education law requiring states to hold schools accountable based on several measures including student achievement. In Tennessee, TNReady provides that measure. State and federal officials have been in talks for weeks trying to navigate both the federal law and the state’s emergency measures. Tennessee’s Department of Education expects to share guidelines with districts soon on how the data should be used at the local level.

“The new legislation impacts many areas,” Gast said, “and our goal is to implement the language as written while honoring the spirit in which it was passed.”

In the meantime, Tennessee will move ahead with scoring the tests through its vendor, Questar. Although the computerized exams were administered in fits and starts, the digital format allows that scoring to be expedited. Paper tests also have been shipped to the Minnesota-based company to be scanned and scored in the coming weeks.

The state plans to share early results for high school end-of-course exams with school districts later this month, and for grades 3-8 by mid-June. Fuller score reports will be distributed to families and districts during the summer.