National testing time

In first comparison with other large cities, Memphis students score poorly on national test

PHOTO: Laura Faith Kebede

Shelby County Schools has gotten its first marks on a national test considered the best way to compare student performance across locations — and the results aren’t good.

Compared to 27 large cities, the district landed in the bottom third of results from the National Assessment of Education Progress, known as NAEP. It also lagged far behind Tennessee’s scores — in which only about one-third of students met the basic standards for proficiency.

The results temper Tennessee officials’ frequently cited claim that the state’s students are improving academically faster than students in other states, but the state’s top education official said she was not surprised by Shelby County’s poor results.

The data was reported Tuesday from NAEP, a test given every two years that measures achievement among a sample of fourth- and eighth-graders across the country. Scores from 27 urban districts are reported separately. This is the first year Shelby County Schools has been compared with the other large urban districts.

NAEP is important because it has long been the only way to compare student performance across states, a distinction that has persisted even as many states adopted similar standards for what students should learn and begun testing students on whether they meet those standards. The exam, which is administered to a sample of students across each state and district, also serves as a check on state test scores.

The Memphis district ranked close to the lowest third in fourth-grade math and reading and in the lowest third in eighth-grade math and reading compared to other cities.

In addition, the scores from Shelby County Schools lagged far behind Tennessee’s scores. Candice McQueen, Tennessee’s top education official, said given Shelby County Schools’ recent performance on the state’s more rigorous test last year, the national city comparison was not surprising.

Superintendent Dorsey Hopson said the results tell him the district needs to stay the course in implementing a myriad of changes to better match instruction to state and national benchmarks.

“I think we’re doing the right work. I think it’s going to take time to implement all the things we want to implement,” he said Tuesday. “To me, it validates why you have to change. Expose kids and prepare kids for the rigor that NAEP measures as well as what the new Tennessee test requires.”

Hopson had said his district was driving the state’s gains over the past several years. He based his claims on the improvements students made on state tests that prior year. It’s still possible that Memphis students propelled the state’s outsized gains on NAEP between 2011 and 2015 — but the new data suggest that local students’ skills are hardly worth bragging about. Shelby County Schools, Tennessee’s largest district, makes up 11 percent of the state’s student population.

When the state introduced a new test in 2017, Shelby County’s performance dropped, along with the state’s. “Shelby County was in some of our lowest proficiency levels across the board, as many of our urban districts,” McQueen said. “So we need to continue to think about what are we doing specifically across all of our urban districts to provide support, resources, and intention to all students.”

The national test is low stakes for individual students and schools, but high stakes for politicians and education leaders who use the scores to tout or denounce education policies. The federal agency that administers the tests warns against connecting scores to specific policies.

A Shelby County Schools statement notes that Memphis fourth graders “surpassed their peers in cities like ours” in which 30 percent or more school-age children live in poverty.

“And with the exception of Atlanta, the cities our students are lagging behind have less than 30-percent of their school-aged children living in poverty, which makes the correlation clear.”

Of the participating school districts, Memphis’ 30 percent poverty rate for children was the fifth highest of the participating urban school districts, according to U.S. Census data. Their test scores mostly correlated with the percentage of families with children living in families, with the exception of eighth-grade reading, in which Memphis students scored higher than their poverty ranking.

Note: The score threshold for “basic” in the charts below signifies partial mastery according to NAEP’s standards.

Update, April 10, 2018: This story has been updated to include a statement from Shelby County Schools, comment from Dorsey Hopson, and more information about city NAEP scores and their poverty rates. 

We created several graphs and charts to examine NAEP scores in Memphis, Tennessee and across cities. Look through them all or skip to the ones you’re most interested in:
2017 NAEP Trial Urban District Assessment Rankings for 4th Grade Math
2017 NAEP Trial Urban District Assessment Rankings for 8th Grade Math
2017 NAEP Trial Urban District Assessment Rankings for 4th Grade Reading
2017 NAEP Trial Urban District Assessment Rankings for 8th Grade Reading
Comparing Memphis, Tennessee and nation: 4th grade reading
Comparing Memphis, Tennessee and nation: 4th grade math
Comparing Memphis, Tennessee and nation: 8th grade reading
Comparing Memphis, Tennessee and nation: 8th grade math
Large City NAEP scores by Poverty: 4th grade reading
Large City NAEP scores by Poverty: 4th grade math
Large City NAEP scores by Poverty: 8th grade reading
Large City NAEP scores by Poverty: 8th grade math

2017 NAEP Trial Urban District Assessment Rankings for 4th Grade Math

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Mathematics Assessments. / Graphic by Sam Park
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Mathematics Assessments. / Graphic by Sam Park

2017 NAEP Trial Urban District Assessment Rankings for 8th Grade Math

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Mathematics Assessments. / Graphic by Sam Park
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Mathematics Assessments. / Graphic by Sam Park

2017 NAEP Trial Urban District Assessment Rankings for 4th Grade Reading

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Reading Assessments. / Graphic by Sam Park
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Reading Assessments. / Graphic by Sam Park

2017 NAEP Trial Urban District Assessment Rankings for 8th Grade Reading

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Reading Assessments. / Graphic by Sam Park
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Reading Assessments. / Graphic by Sam Park

Comparing Memphis, Tennessee and nation: 4th grade reading

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Reading / Mathematics Assessments.

Comparing Memphis, Tennessee and nation: 4th grade math

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP).

Comparing Memphis, Tennessee and nation: 8th grade reading

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP).

Comparing Memphis, Tennessee and nation: 8th grade math

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP).

Large City NAEP scores by Poverty: 4th grade reading

Large City NAEP scores by Poverty: 4th grade math

Large City NAEP scores by Poverty: 8th grade reading

Large City NAEP scores by Poverty: 8th grade math

new rules

Now that TNReady scores will count less for students, will they even try?

PHOTO: Nicholas Garcia

In the face of a statewide testing debacle, the Tennessee legislature’s hasty edict this week to discount test results has mollified some teachers and parents, but raised more questions about the role of test scores and further eroded the motivation of students, who must labor for about two more weeks on the much-maligned TNReady test.

Thursday’s sweeping measure to allow districts to ignore test results when grading students and to prohibit the use of test scores when determining teacher compensation has left educators and students shrugging their shoulders.

“I’ve gone from ‘oh well, tests are just a part of life’ to ‘this is an egregious waste of time and resources and does not respect the developmental needs of our children,’” said Shelby County parent Tracy O’Connor. For her four children, the testing chaos has “given them the idea that their school system is not particularly competent and the whole thing is a big joke.”

Her son, Alex O’Connor, was even more succinct. “We spend $30 million on tests that don’t work, but we can’t get new textbooks every year?” said the 10th-grader at Central High School. “What’s up with that? I’m sure half of us here could design a better test. It’s like buying a used car for the price of a Lamborghini.”

The legislature’s decision created a new challenge for Tennessee’s Department of Education, which planned to use 2018 TNReady testing data to rate and identify the lowest-performing schools, as required by the federal government. Now, with the test’s reliability under question, state officials say they are determining “additional guidance” to provide districts on how the state will comply with the U.S. Department of Education.

Student test results still will be used to generate a score for each teacher in the Tennessee Value-Added Assessment System, known as TVAAS. Scores will count for 20 percent of teachers’ evaluations, though districts now cannot use the scores for any decisions related to hiring, firing, or compensating teachers.

For students, local school boards will determine how much TNReady scores will count toward final grades — but only up to 15 percent. Several school districts have already expressed serious reservations about the testing data and likely won’t use them in students grades at all. And in previous years, the results didn’t come back in time for districts to incorporate them anyway.

In sum, asked Memphis sophomore Lou Davis, “Why are we doing this anymore when know it won’t count?”

About 650,000 students are supposed to take TNReady this year, with 300,000 of them testing online, according to the state. Each student takes multiple tests. As of Friday, more than  500,000 online tests sessions had been completed.

Even as testing continues, some education leaders worry the exam’s credibility is likely to sink even further, because students might not try, and parents and teachers may not encourage much effort.

“In the immediate term, there’s concern about how seriously people will take the test if they know it’s not going to count,” said Gini Pupo-Walker, head of the Tennessee Educational Equity Coalition and a member of the state’s testing task force. “Will students continue to take the test? Will kids show up? Will parents send their kids to school?” she asked. “Now, there’s the whole question of validity.”

Sara Gast, spokeswoman for the Department of Education, said while the new legislation provides more flexibility for districts in how they use TNReady results, it doesn’t mean that the results don’t matter.

“The results always matter. They provide key feedback on how students are growing and what they are learning, and they provide a big-picture check on how well they are mastering our state academic expectations,” Gast said. “It serves as accountability for the millions of taxpayer dollars that are invested into public education each year.”

Jessica Fogarty, a Tullahoma school board member and parent, says she thinks this year’s testing issues could lead to more parents telling their kids to refuse state tests in the future.

A proponent of opting out of state tests, Fogarty said, “We need to understand that we can choose what our children do or do not suffer through. I hope this debacle showed parents what a waste of time this is — students would gain more through reading a book.”

Because Tennessee has no official opt-out policy, students wanting to opt out must “refuse the test” when their teacher hands it to them.

Jessica Proseus, a parent of a student at Bartlett High School, said her daughter has opted out of state testing in the past, but started taking the exams this year because she believed it could affect her final grades.

“With college looming in a couple years, she couldn’t afford to get zeroes on her report cards,” Proseus said. But with the test debacle, her daughter might change her mind and just skip the remaining two weeks of testing.

“I even took the online practice TNReady a few years ago and it was terribly confusing to navigate,” Proseus said. “The testing in Tennessee is not transparent — it is almost like it is set up to trick and fail children — and that’s very cruel for a young child to deal with.”

Chalkbeat explains

Four reasons Tennessee likely won’t go back to paper testing

As another wave of problems with online testing plague Tennessee schools, one of the solutions proposed by state legislators — go back to paper exams — is a stretch for a state that has invested millions into electronic exams.

In short, reverting to pencil-and-paper tests would be akin to ordering iPhone users to go back to flip phones. It almost certainly won’t happen.

Two Memphis-area state lawmakers want to ban the online version of TNReady starting next school year until the state comptroller determines its problems are “fully and completely fixed.” And other lawmakers suggest districts should be able to choose between paper and electronic testing..

(Other amendments that would ensure this year’s test results wouldn’t count against teachers, students, or schools passed Thursday.)

The list of problems has grown since the first day of testing Monday, affecting about two dozen districts, including the four largest ones in Tennessee. The meltdown follows the monumental online failure in 2016 when a server crash prompted Education Commissioner Candice McQueen to cancel most of state testing that year.

Here are four reasons why it’s unlikely Tennessee won’t go back to paper testing despite current overwhelming frustrations:

Superintendents think they’ve gone too far to turn back now. Maryville Director of Schools Mike Winstead cautioned against rash decisions in the heat of the moment.

“When things like this happen, it’s easy to overreact,” he told Chalkbeat. “But we’ve come too far. We know that online testing is the future. If we turn back, it will take a long time to get back to where we were.”

And school systems and counties have poured millions into infrastructure and devices, said Dale Lynch, the executive director of the Tennessee Organization of School Superintendents.

“We don’t want to back up. We want to get it right, though,” he said.

Paper is more time consuming. With online testing, McQueen said Wednesday, “we can get test materials [and scores] back or to folks much quicker.”

Preparing paper tests requires hours of sorting and labeling exams. And if the materials arrive late, like they did for several districts this month because of severe weather at Questar’s printing center in the Northeast, the time crunch is especially stressful.

Granted, a top-notch online system that protects against cheating and hacking could be more expensive than a paper version, said Wayne Camara, the research chair at ACT who has long overseen test security.

“The issue of cost is relative.” he said. Multiple versions of computer tests are necessary to help safeguard against cheating, especially via social media.

“If you’ve having to produce 10 or 15 forms of a computer test, most likely it’s not cheaper.”

If Tennessee switches back to paper testing, it will be one of few states nationwide. A recent analysis by John Hopkins School of Education listed 11 states that were still using paper tests in 2016 for elementary students. For middle schools, it was nine states.

Nearly across the board, those states with no experience with online testing did worse in national online testing.


Read more about Tennessee’s most recent performance on “the nation’s report card.”


There’s security issues with paper too. The alleged cyber attack on Questar’s data center Tuesday spraked a statewide outcry, but switching back to paper won’t eliminate security issues.

“Both digital- and paper-based testing are certainly susceptible to cheating,” said Camara, the testing cybersecurity expert. “I don’t think anybody would say that there’s a significant reduction of security measures or cheating with computers, it’s just different.”

One of the largest state test cheating scandals happened in Atlanta with paper tests when principals and teachers changed student answers. That’s much harder to do online.

Jacinthia Jones and Marta W. Aldrich contributed to this story.