Switch glitch

Tennessee’s switch to online testing starts rough as platform crashes on Day One

PHOTO: J. Zubrzycki
Computers line a classroom at Aspire Hanley Elementary in Memphis as schools prepared to make the switch to online assessments for state achievement tests.

On the first day of online testing in Tennessee schools, a network outage forced students to stop taking the state’s new achievement test, causing a delay in the much-anticipated rollout of the TNReady exam.

Major outages occurred statewide at 8:25 a.m. CST on the MIST platform developed by Measurement Inc., the North Carolina-based company that created the test, according to an email to superintendents from state Education Commissioner Candice McQueen.

“These outages were caused because the network utilized by Measurement Inc. experienced a failure,” McQueen wrote. “We are urgently working with Measurement Inc. to identify the causes and correct the problem. At this time, we are advising that schools experiencing problems with the test discontinue testing, and return to their normal classes. Please do not begin any new additional testing you had planned for today until the department provides further information.

Due to a lengthy testing window that began on Monday and continues through March 4, it’s unclear how many schools were impacted statewide.

However, in Nashville, 30 schools were administering the test on Monday and six reported glitches, according to Joe Bass, a spokesman for Metropolitan Nashville Public Schools.

Students who had not yet encountered technological difficulties were told to finish the assessment.

District leaders stressed to teachers and parents that the outages were not due to local technology problems. McQueen told superintendents that Measurement Inc. was aware of the problem.

Technology troubles have plagued several states in recent years in the transition to online testing, causing mistrust over the validity of scores and testing platforms.

As word spread about Monday’s outages, critics of the new test — and the inclusion of data from the tests in teacher evaluations — quickly cited the technology glitches as an indictment of the State Department of Education.

#TNReady crashes statewide … is TDOE still perplexed why boards across the state are asking for grace for teachers and kids from scores?” Knox County school board member Amber Rountree tweeted upon learning the news.

Tennessee Education Association President Barbara Gray quickly blasted the state Education Department’s handling of the transition, citing concerns about the state’s capacity to accommodate so many students on the server at one time, as well as concerns about local districts having enough resources to complete the testing.

“It is unacceptable to have this kind of statewide failure when the state has tied so many high-stakes decisions to the results of this assessment,” Gray said. “Our students and teachers have enough stress and anxiety around these assessments without adding additional worries about technical issues.

Gray called on the state to grant at least a one-year waiver from including TNReady scores in teacher evaluations, although state officials have said that districts have discretion whether they choose to use that data this year on personnel decisions.

Last week, Department of Education officials told reporters they were prepared for widespread technology difficulties.

Chief Information Officer Cliff Lloyd said that he wasn’t as concerned about major failures such as servers crashing. “Those big things are easy to fix,” he said at the time. “I’m worried about small things that can occur on certain operating systems in certain conditions.”

As part of the state’s contingency plans, districts can request paper-based tests if technical problems persist and also can administer TNReady later than the state-assigned testing window.

 

Editor’s note: This story has been updated from an earlier version.

TNReady backlash

Tennessee lawmakers take matters into their own hands on TNReady testing problems

PHOTO: Marta W. Aldrich
State lawmakers are in session at the Tennessee State Capitol in Nashville.

It was an extraordinary day on Capitol Hill in Nashville and, in many ways, unprecedented.

As reports of more problems with Tennessee’s standardized test escalated from their public schools back home, members of the General Assembly overwhelmingly approved a sweeping measure to pull this year’s TNReady scores from accountability systems for students, teachers, schools, and districts.

A spokeswoman said Gov. Bill Haslam will sign the legislation.

The votes circumvented the legislature’s committee process but, after days of technical problems with the state’s return to online testing, lawmakers had reached a boiling point. In the midst of an election year, they rose to their feet and, one after another, railed against the Department of Education and its testing company, Questar, for their oversight of the beleaguered test.

Get more stories like this in your inbox!
Sign up for Chalkbeat newsletters here, and get the education news you care about delivered daily.

At midday, the Senate and House convened a conference committee as a bipartisan coalition of House members used passage of the state’s $37.5 billion budget as a bargaining chip. With lawmakers going back and forth to the governor’s office to confer, they tacked on their amendment to a bill sponsored by Rep. Eddie Smith of Knoxville and Sen. Dolores Gresham of Somerville.

“The camel was already loaded down heavy, but this was the straw that broke the camel’s back,” Smith said of more testing glitches on Thursday. “The circumstances were so extraordinary that going through the traditional committee process did not serve our teachers or students. That’s why we did what we did.”

What they did was pass a bill to:

  • Let local school boards determine, between a range of 0 and 15 percent, what TNReady scores will count toward students’ final grades;
  • Prevent local districts from using the scores for any decisions related to hiring, firing, or compensating teachers;
  • Ensure that none of this year’s TNReady data can be used to put a school on the “priority list” of lowest-performing schools eligible for state intervention; and
  • Nix the use of TNReady data in determining A-F ratings for schools, a system that’s to begin this fall

“It was clear many members of the General Assembly wanted to address concerns related to the recent administration of state assessments,” Haslam spokeswoman Jennifer Donnals said in a statement. “The governor understands these concerns and did not oppose the legislation.”

The decision means Tennessee will take a breath as it seeks to fix its broken testing system, which has been snakebit from the outset. In 2016, Education Commissioner Candice McQueen canceled most testing after TNReady’s new online platform collapsed under the weight of statewide testing on newly minted digital devices. The next year, Tennessee reverted to mostly paper-and-pencil tests, but there were scoring and score delivery issues under new vendor Questar.

This week, when the third year of testing launched, McQueen had been more confident under a gradual transition to online testing beginning with high school students. But on Monday, a login issue stopped testing in its tracks. Tuesday was worse, as Questar’s system shut down because of an alleged cyber attack.

“What you heard today is that, until we get testing right, we want to make sure our teachers, students and schools are not impacted,” Smith told Chalkbeat when the dust had settled on Thursday.

“We’re still going to move forward with our accountability system. We’ll still see what the data shows this year. But we want to make sure the data isn’t skewed. We want to make sure it’s reliable.”

News of the pause drew immediate cheers from teacher groups.

“The legislature made sure students, teachers and schools were protected against the failures of TNReady,” said Jim Wrye, a lobbyist for the Tennessee Education Association, commending lawmakers for taking “decisive action.”

“We are very pleased legislators ensured that employment or compensation decisions based on the data cannot be used,” added JC Bowman, executive director of Professional Educators of Tennessee.

But others warned that systems for holding teachers and schools accountable are key to ensuring an equitable education for all students.

“While we are dismayed that there were issues with the online TNReady tests, we believe that assessments are the clearest way to gauge what students know, and how well schools are serving all students,” said Gini Pupo-Walker of the Tennessee Educational Equity Coalition.

“Tennessee has made great progress by raising expectations, creating high standards and implementing TNReady,” Pupo-Walker said, “and it is important to continue to assess students every year on their mastery in the core content areas.”

Correction: April 19, 2018: This story has been corrected to show that both the Senate and the House approved the bill on Friday. A previous version said the Senate had recessed and would vote Monday on the House-approved bill.

exceptions to the rule

The highs and lows of Colorado education are spotlighted in ‘The Outliers’

PHOTO: Chalkbeat file
Students at Sheridan's Fort Logan Elementary collaborate on a literacy lesson.

The Boulder Valley School District serves a largely affluent population with highly educated parents. In Sterling on the Eastern Plains, fewer than 1 in 6 adults has a bachelor’s degree. But both the Boulder district and the Valley Re-1 district serving Sterling send a large portion of their graduates to college, and few of them need remediation classes when they get there.

Those are just two of the findings in a new report from the Denver-based education reform advocacy group A Plus Colorado that examines both exemplary and struggling districts. A Plus focuses on data analysis to drive public support for policy changes. This is the second year that A Plus has released “The Outliers,” which is intended to help educators find models to emulate.

The report notes success stories like DSST: Stapleton High School, part of the Denver-based charter network, which posted the state’s highest average SAT scores for white students, black students and students from low-income families. Its Hispanic students also posted SAT scores that were among the highest in the state.

Among the report’s nuanced findings, the tiny Sheridan district south of Denver sends relatively few students to college, with many later needing remediation. But the district has made big strides in the graduation rate of homeless students, who make up 25 percent of its students. In 2016, 64 percent of its homeless students graduated, compared with 53 percent for the state and 42 percent in Denver.

Here are four takeaways from the report:

The numbers only tell so much.

The report shows schools where students from low-income families — as measured by free- and reduced-price lunch rates — do well on elementary math tests or middle school language arts, and where Hispanic students graduate at high rates or have good SAT scores. However, it doesn’t explain just how those schools succeed.

CEO Van Schoales said A Plus Colorado isn’t able to visit all districts and schools to research what they’re doing right, but he hopes the report can still be a resource for principals and superintendents.

“Folks need to spend the time to get to understand the places where most kids are getting to standards or graduating or showing growth,” he said. “There can be an echo chamber in education around the cool places or what’s hot. This report is the data. There are a lot of places doing great work.”

Small districts are just as capable of serving at-risk students as large ones.

The first year of “The Outliers” only looked at the 76 districts serving at least 1,000 students. This year, the report looks at roughly 120 districts with publicly available data. Small districts are more likely to be outliers in both good and bad ways. With fewer students overall, it doesn’t take many students to significantly boost or drag down achievement percentages.

The researchers found small school districts serving low-income and diverse student populations and getting good outcomes.

“A lot of school districts think that the bigger you are, the more capacity you have and the more good you can do, and our report shows that that is not necessarily true,” Schoales said. “You can find school districts doing well by low-income kids all over Colorado and ones that are not.”

Online schools need a lot more scrutiny.

The report found that students in online schools do worse even than students in low-performing brick-and-mortar schools, and that when districts open online schools, it pulls down districtwide performance.

The Byers district east of Denver, where 82 percent of students attend online schools, and Colorado Digital BOCES, a cooperative collection of online schools, showed some of the lowest academic growth in the state, the report found.

“In theory, it sounds great,” but most online schools are not working for their students, Schoales said.

This is not a new concern. A 2016 investigation by Education Week raised serious questions about the operation of GOAL Academy online schools. But the sector continues to expand, with A Plus calling it “one of the fastest growing segments within the Colorado educational ecosystem.”

Colorado suppresses so much student data that it’s hard to get a complete picture.

Colorado has strict data privacy rules that lead to the suppression of student achievement information from small batches and sometimes even larger groups of students. As a result, A Plus Colorado said it doesn’t know whether the 300 black students in the Boulder Valley School District or the 366 students in Manitou Springs who qualify for free- and reduced-price lunch are meeting grade-level expectations.

That’s because the state redacts scores whenever fewer than four students score at a particular proficiency level, and then shields additional scores from other groups and even other schools to further obscure the data.

A Plus says this 3-year-old policy makes it impossible to discern how certain groups of students are performing.

Complete test data is available to district officials, but Schoales said that’s not good enough. If they and the public can’t compare among school districts, they don’t know how much better they could be doing.

“The public and policy makers need to know what’s working and where we can learn from,” Schoales said.

Read the rest of the report, with lots of district- and school-level information, here: