TNReady Fallout

Tennessee freezes testing contract as online glitches derail TNReady debut

PHOTO: Grace Tatter
Tennessee Education Commissioner Candice McQueen speaks with reporters in February about technical problems with the state's new online assessment.

One day after technical failures crippled Tennessee’s long-awaited switch to online testing, Education Commissioner Candice McQueen put the blame on the test’s developer and said the state is reviewing its $108 million contract with Measurement Inc.

“We have doubts about them going forward, and yes, we have concerns, and yes, we are reviewing that currently,” McQueen said during a news conference on Tuesday morning, one day after directing school districts to scrub Tennessee’s new online assessment and stick with a paper-and-pencil version for now.

“Our expenditures to Measurement Incorporated are based on what’s actually delivered, and today we don’t have an online platform,” said McQueen, adding that the state has paid the North Carolina vendor only $1.6 million so far.

A relatively small testing company in Durham, Measurement Inc. was awarded the contract in late 2014 to develop Tennessee’s new test, dubbed TNReady, to provide an online platform, as well as to align the assessment with the state’s current Common Core academic standards. Tennessee opted to use a private vendor after its legislature jettisoned the PARCC test, which was created in collaboration with several other states but criticized as federally intrusive because it wasn’t Tennessee-specific.

Measurement Inc. had worked with the state previously to develop a writing test for grades 3-11. And of five vendors bidding for the bigger task, the company stood out, according to state officials.

“It was truly the one that was immediately pointed to because it was the one with the lowest cost and the highest score,” McQueen said.

But on Monday at 8:25 a.m. CST, only minutes after students began using the online testing platform developed by Measurement Inc., a network outage forced students to stop taking the state’s new achievement test, the result of years of development, preparation and testing. By the end of the day, McQueen and her leadership team made to call to scrap the online transition for the school year.

“The new nature of the issue yesterday highlighted the uncertainty around this platform,” McQueen said Tuesday. “Despite the many improvements the department has helped make to the system in recent months, we are not confident in the system’s ability to actually perform consistently.”

Measurement Inc. president Henry Scherich says it was unneccessary for state officials to pull the plug on the online platform and noted that nearly 20,000 Tennessee students successfully completed their assessments on Monday. “Although MI believes that the server overload problem has been corrected, the State made the decision to discontinue online assessments,” he said in a statement released on Tuesday afternoon.

Many states have experienced glitches in their switch to online tests. For the most part, students in states who used PARCC, as Tennessee originally planned, were able to complete their tests online, though they scored lower than their peers who took the PARCC with pencil and paper.

But none of those states’ technical problems appear to have been as widespread as Tennessee’s breakdown — or as far-reaching in its fallout. McQueen’s decision to completely abandon online testing on the first day of the state’s new assessment was jarring and leaves Tennessee education leaders scrambling to assure teachers, students and parents of the test’s accuracy.

“PARCC, as far as I know, had no problems,” said Scott Marion, the director of the National Center for the Improvement of Educational Assessment and a technical advisor for PARCC. “(Tennessee was) on a very fast time schedule from when you left PARCC to when you had to develop a new assessment, and that is always a bit of a challenge.”

Advocates for fair testing note that system failures such as Tennessee’s also occurred in Florida, with other states experiencing significant disruptions in teaching and learning due to the switch to automated assessments.

“The collapse of Tennessee’s new computer testing system on the first day of administration is neither unexpected or unprecedented,” said Bob Schaeffer, public education director for the National Center for Fair & Open Testing, a nonpartisan group monitoring the use of standardized tests.

“Across the country, dozens of jurisdictions have experienced similar technical issues in attempting to introduce automated assessments. … The reason for this spate of problems is that computerized testing is being rushed into the marketplace before it is ready for prime time. Rather than heeding the advice of technology or education experts (or the experience of other states), politicians and ideologues have demanded artificial implementation timetables that do not allow sufficient time to develop the necessary infrastructure,” he said.

McQueen expressed confidence that the shift to TNReady was the right thing to do. Calm and smiling throughout Tuesday’s news conference, she also maintained that Tennessee’s current law  factoring data from the test into teacher evaluations is fair because of the state’s agreement to temporarily lower the weight of this year’s test in evaluations — although not totally negating it, as many educators wanted.

”That was truly about technology, to make sure that we didn’t have a technology transition that could harm someone, but only help them,” McQueen said.

She said that the paper version of TNReady, also developed by Measurement Inc., is fully aligned to “the depth and the breadth” of Tennessee’s standards and will give parents, students and teachers better information about students’ readiness for postsecondary education or training.

“TNReady remains, TNReady continues,” she said.

And though no timeline is in place, McQueen expects Tennessee eventually will join the ranks of states administering online tests, meaning that local districts’ investment in time and money for technology for the new test is not in vain.

“The investment has been a good one. The investment has been positive,” she said. “We’re still moving online. It’s the way of the world.”

 

Editor’s Note: This story has been updated to include a response from the president of Measurement Inc.

Testing Testing

“ILEARN” is in, ISTEP is out — Indiana legislature approves test set to begin in 2019. Now awaiting governor’s OK.

PHOTO: Grace Tatter

A little more than a year ago, lawmakers made the dramatic call to “repeal” the state’s beleaguered ISTEP test without a set alternative.

Friday night, they finally decided on a plan for what should replace it.

The “ILEARN” testing system in House Bill 1003 passed the House 68-29 and passed the Senate 39-11. Next, the bill will go to Gov. Eric Holcomb for him to sign into law.

The new test would be used for the first time in 2019, meaning ISTEP still has one more year of life. In the meantime, the Indiana Department of Education will be tasked with developing the new test and finding a vendor. Currently, the state contracts with the British test writing company Pearson.

House Speaker Brian Bosma said he was very pleased with the compromise, which he thinks could result in a short, more effective test — although many of those details will depend on the final test writer.

However, a number of Democrats, and even some Republicans, expressed frustration with the testing proposal.

“The federal government requires us to take one test,” said Sen. Aaron Freeman, a Republican from Indianapolis. “Why we continue to add more and more to this, I have no idea.”

For the most part, the test resembles what was recommended by a group of educators, lawmakers and policymakers charged with studying a test replacement. There would be a new year-end test for elementary and middle school students, and High schools would give end-of-course exams in 10th grade English, ninth-grade biology, and algebra I.

An optional end-of-course exam would be added for U.S. government, and the state would be required to test kids in social studies once in fifth or eighth grade.

It’s not clear if the plan still includes state Superintendent Jennifer McCormick’s suggestion to use an elementary and middle school test that would be “computer-adaptive” and adjust difficulty based on students’ answers.

The plan does make potentially significant changes to the state’s graduation requirements. Rather than having ECAs count as the “graduation exam,” the bill would create a number of graduation pathways that the Indiana State Board of Education would flesh out. Options could include the SAT, ACT, industry certifications, or the ASVAB military entrance exam.

Test researchers who have come to speak to Indiana lawmakers have cautioned against such a move, as many of these measures were not designed to determine high school graduation.

While teacher evaluations would still be expected to include test scores in some way, the bill gives some flexibility to districts as to specifically how to incorporate them, said Rep. Bob Behning, an Indianapolis Republican and the bill’s author.

Currently, law says ISTEP scores must “significantly inform” evaluations, but districts use a wide range of percentages to fit that requirement.

You can find all of Chalkbeat’s testing coverage here.

lingering debate

Drop TNReady scores from teacher evaluations, urge Shelby County leaders

PHOTO: The Commercial Appeal
From left: Commissioners Reginald Milton, Van Turner and David Reaves listen during a meeting in Memphis of the Shelby County Board of Commissioners. The governing board this week urged state lawmakers to strip TNReady scores from teacher evaluations.

Just as students have begun taking Tennessee’s new standardized test, Shelby County officials are calling on state leaders to back off of using those scores to evaluate teachers.

The Shelby County Board of Commissioners, the local funding body for Memphis schools, voted unanimously on Monday to urge  the state to use TNReady results as only a “diagnostic” tool. Currently, the board says, state scores are being used as a punitive evaluation of both teachers and students.

The board’s call gets to the heart of a debate that has lingered since a 2010 state law tied standardized test results to teacher evaluations. That was several years before TNReady was introduced last year as a new measuring stick for determining how Tennessee students — and their teachers — are doing.

TNReady testing, which began this week and continues through May 5, has intensified that debate. The new test is aligned to more rigorous academic standards that Tennessee is counting on to improve the state’s national ranking.

But Shelby County’s board is questioning whether reforms initiated under Tennessee’s 2010 First to the Top plan are working.

“While giving off the appearance of a better education, this type of teaching to the test behavior actually limits the amount of quality content in deference to test taking strategies,” the board’s resolution reads.

The board also cites “unintended consequences” to the teaching profession as nearly half of Tennessee’s 65,000 teachers are expected to leave or retire in the next decade.

“Record numbers of quality teachers are leaving the teaching profession and school districts are struggling to recruit and retain quality teachers due to the TN standards imposed in regards to standardized testing,” the resolution reads.

It’s true that school districts statewide struggle to recruit and retain effective teachers in some subject areas. But there’s little evidence to support that incorporating test scores in evaluations is the primary reason teachers are leaving the profession.

It’s also unlikely that Tennessee will back off of its teacher evaluation model, even as some states have recently abandoned the practice. The model is baked into reforms that the state initiated through two gubernatorial administrations to improve both teacher and student performance.


Want education equity? Make sure your teachers feel valued, say lawmakers


PHOTO: Yalonda M. James/The Commercial Appeal
Commissioner David Reaves

Shelby County’s resolution was introduced by Commissioner David Reaves, a former Memphis school board member who says he hears a “continual outcry” from teachers and parents over high-stakes testing.

“Allow the local (school district) to assess and classify teachers and use the test results as a tool, not as a stick,” Reaves told Chalkbeat.

In Tennessee, test scores in some form count for 35 to 50 percent of teachers’ evaluation scores. TNReady scores currently count 10 percent but, as the state settles into its new test, that will gradually increase to 25 percent by 2018-19.

Classroom observations and evaluations did play a factor in retention rates for effective teachers in a 2014 study by the Tennessee Department of Education before the transition to TNReady. Where teachers reported consistent and objective classroom observations, effective teachers were more likely to stay.

State and local teacher surveys also differ on the quality of Tennessee’s teacher evaluation system known as TEAM, which mostly relies on classroom observations.

In Shelby County Schools, exit surveys show issues like levels and stability of teacher pay — not test scores in their evaluations — are cited most often by teachers leaving the district.

Superintendent Dorsey Hopson told the school board last month that most Shelby County teachers find the state’s evaluation system unfair, but the same majority think their own score is fair.

Another survey by the Tennessee Department of Education suggests that satisfaction with the state’s evaluation system is on the rise as teacher feedback continues to be incorporated.

The Shelby County board, which oversees funding for Tennessee’s largest district, is sending its resolution to Gov. Bill Haslam, Education Commissioner Candice McQueen, and the Tennessee General Assembly. Below is the full text: