MIA

Some Tennessee students lost their TNReady test answers despite state’s reassurance they wouldn’t

PHOTO: Nicholas Garcia

The frequent technical interruptions that marred online testing in Tennessee this year were bad enough for Cathy Palmer, a teacher at Central High School in Memphis. But students losing their test answers — despite the state’s assurance they wouldn’t — was just too much.

She recalled a case of a student who logged in to the second part of her history test April 17 and discovered the first part she completed the previous day was marked as incomplete.

“We were told that if this happened, we should make sure the student was using the same computer, sign in to that particular part, and submit the test. When we did this for her, her entire essay was gone,” Palmer said.

In most cases where tests didn’t submit properly, students were able to log back in later and submit their tests, according to the state. It was inconvenient, but the answers were stored just as the state’s test maker, Questar, said they would be.

But in an unknown number of cases statewide, students lost test answers. Chalkbeat interviewed teachers and administrators from three school districts who said students lost work. Sara Gast, a spokeswoman for the Tennessee Department of Education, said the test recovery process worked “in the vast majority of cases” but, when asked, did not provide the number of instances where it did not.

“In some cases, Questar’s team needs additional time to recover these tests, and they are still working through those,” she said last week.

The complication raises further questions about how well this year’s online state test, known as TNReady, will show what students know. All high school students were required to take the online version this year, and 59 districts opted to offer it to middle school students. The rest of Tennessee’s third- through-eighth-graders are taking TNReady on paper.

The state’s online system experienced widespread interruptions on at least four days since testing began April 16. There were log-in issues on the first day, a reported cyberattack on the second, and a problem with lists of students taking the test later in the week after the state’s testing company, Questar, updated its software the night before. An issue with a feature that reads test questions aloud for students with disabilities also caused problems for students trying to login and submit their tests.

Tennessee’s largest district, Shelby County Schools, said this has happened “several days” during the testing window, but did not provide specific numbers either.

“We do not have an exact count. In some cases, we are still waiting to see if the recovery process works,” the district said in a statement.

“The recovery process can take a few days, and therefore answers can still appear to be missing even though they are being recovered,” the district said in another statement. “If the recovery process fails for a student, we have been advised to consult with the Tennessee Department of Education on the best way to proceed.”

In Haywood County, two students lost their test answers and their parents decided not to retest them, a decision the superintendent, Joey Hassell, supported.

Students losing their test answers has been an issue throughout the testing window, said Rachel Phillips, a teacher in Sumner County. It happened to three of her students. She said Questar’s helpline has not been able to explain why it happens.

“They say, ‘I don’t know’ a lot of the time. When we push them and say that is unacceptable, they complete a ‘ticket’ and say they will send directions for how to recover the test. Then you follow the directions for recovered individual student’s tests. Or attempt to,” Phillips said. “The students get very frustrated and nervous that all the work they just did on the test is gone.”

Rick Eaton, the testing coordinator for Sumner County, said the district is still compiling the total number of students affected, but that all middle and high schools were impacted “to some extent.”

“Each of those content areas have 3 to 4 subparts to them. If you experience something in one subpart and an irregularity report for a subpart, the entire content area is invalidated,” he said.

Gast, the state spokeswoman, said local school districts have discretion on whether or not the student should retake the test in those instances. Districts can also submit a report for the online issues “if they believe those disruptions impacted students’ ability to show their knowledge of the standards,” which would invalidate the students’ test results.

Palmer said the student who lost her essay was told to retake that part of the test.

“About 10 minutes later, I noticed that tears were falling down her face. I asked if she was OK, and she said she was willing to re-write her essay, but, in her words, ‘I just can’t do it today,’” Palmer said. “It took me about two seconds to make the decision and tell her to log out. She did not need to be tortured like that.”

“Everyone in my school is very frustrated. Teachers feel frantic and confused amidst all of this chaos,” she continued.

State lawmakers passed two bills to mitigate the impact of this year’s test results on student grades, teacher evaluations, and school accountability. But state officials are still interpreting what that means for its plan to comply with federal law, which requires test scores in part to fuel decisions about schools.

Trending up

Most schools in Tennessee’s largest district show growth on state test

PHOTO: Laura Faith Kebede
Students at Freedom Preparatory Academy's high school prepare to take their TNReady geometry test.

Most schools in Shelby County Schools showed progress in all subjects except science, but students still outshined their peers across the state in science, earning them the state’s highest rating in growth.

About half of schools in the Memphis district saw a bump in English scores, also earning the district the highest rating of growth under the Tennessee Value-Added Assessment System, known as TVAAS.

Superintendent Dorsey Hopson attributed the gains to a renewed focus in preschool education in recent years, adding a reading curriculum more aligned with state standards, and doubling down on literacy training for teachers and students.

“When you think about the investments that we’ve been able to make in schools over the last two years, I think the data is showing that we’re seeing a good return on our investment,” he told reporters Thursday.

But the scores don’t come without tension. Hopson recently teamed up with Shawn Joseph, the director of Metro Nashville Public Schools, to declare “no confidence” in the state’s test delivery system, which has been plagued with online problems since it began in 2016. Still, Hopson said educators are utilizing the data available to adjust strategies.

“It’s an imperfect measure, but it’s the measure we have right now,” he said. Hopson worries the failures of the state’s online testing system used by high schoolers made “some teachers and students lose focus.”

“There’s impact on those kids that we may never know about,” he said.

Find your school and compare here

The state doesn’t release data for an exam if fewer than 5 percent of students performed on grade level or if 95 percent of students were above grade level. An asterisk signifies that a school’s score falls in one of those two categories.

District-wide results released in July show that more young students are reading on grade-level, and that math scores went up across the board. But the percentage of high school students who scored proficient in reading dropped by 4 percentage points. Shelby County Schools still lags significantly behind the state average.

Shelby County Schools also improved its overall growth score, which measures how students performed compared to peers across the state who scored similarly to them the year before. It increased from 1 to 2 on a scale of 5. More than half of schools scored 3 or above, meaning those students scored on par or more than their peers.

The district’s nearly 200 schools include about 50 charter schools that are managed by nonprofit organizations but receive public funding. The rest are run by the district.

Below are charts showing the five schools that performed best and worst in the district in each subject, as well as those that grew or declined the most in each subject.

The state doesn’t release data for an exam if fewer than 5 percent of students were on grade level or if 95 percent of students were above grade level. The charts below only include schools that fall in between that range.

English Language Arts

Graphic by Samuel Park
Graphic by Samuel Park
Graphic by Samuel Park
Graphic by Samuel Park

Math

Graphic by Samuel Park
Graphic by Samuel Park
Graphic by Samuel Park
Graphic by Samuel Park

Science

Graphic by Samuel Park
Graphic by Samuel Park
Graphic by Samuel Park

Graphic by Samuel Park

Social Studies

Graphic by Samuel Park
Graphic by Samuel Park
Graphic by Samuel Park
Graphic by Samuel Park

colorado accountability

Test results can spell relief or gloom for state’s lowest performing schools and districts

File photo of sixth-grade students at Kearney Middle School in Commerce City. (Photo by Craig Walker, The Denver Post)

All three school Colorado districts under the gun to improve their academics showed some gains on test results released Thursday — but the numbers may not be enough to save one, Adams 14, from facing increased state intervention.

Of the three districts, only the Commerce City-based Adams 14 faces a fall deadline to bump up its state ratings. If the district doesn’t move up on the five-step scale, the state could close schools, merge Adams 14 with a higher-performing neighbor, or order other shake-ups.

The school district of Westminster and the Aguilar school district, also on state-ordered improvement plans, have until 2019 to boost their state ratings.

The ratings, expected in a few weeks, are compiled largely from the scores released Thursday which are based on spring tests.

District officials in Adams 14 celebrated gains at some individual schools, but as a district, achievement remained mostly dismal.

“We continue to see a positive trend in both English language arts and math, but we still have work to do,” said Jamie Ball, manager of accountability and assessment for Adams 14.

The district’s high school, Adams City High School, which has its own state order to improve its ratings by this fall, posted some declines in student achievement.

District officials said they are digging into their data in anticipation of another hearing before the State Board of Education soon.

In a turn likely to invite higher scrutiny, district schools that have been working with an outside firm, Beyond Textbooks, showed larger declines in student progress.

In part, Ball said that was because Beyond Textbooks wasn’t fully up and running until last school year’s second semester. Still, the district renewed its contract with the Arizona-based firm and expanded it to include more schools.

“Its a learning curve,” said Superintendent Javier Abrego. “People have to get comfortable and familiar with it.”

For state ratings of districts and high schools, about 40 percent will be based on the district’s growth scores — that’s a state measurement of how much students improved year-over-year, when compared with students with a similar test history. A score of 50 is generally considered an average year’s growth. Schools and districts with many struggling students must post high growth scores for them to get students to grade level.

In the case of Adams 14, although growth scores rose in both math and English, the district failed to reach the average of 50.

Credit: Sam Park
PARCC, district on state plans
Credit: Sam Park

Westminster district officials, meanwhile, said that while they often criticize the state’s accountability system, this year they were excited to look at their test data and look forward to seeing their coming ratings.

The district has long committed to a model called competency-based education, despite modest gains in achievement. The model does away with grade levels. Students progress through classes based on when they can prove they learned the content, rather than moving up each year. District officials have often said the state’s method of testing students doesn’t recognize the district’s leaning model.

“It’s clear to us 2017-18 was a successful year,” said Superintendent Pam Swanson. “This is the third year we have had upward progress. We believe competency-based education is working.”

The district posted gains in most tests and categories — although the scores show the extent of its challenge. Fewer than one in five — 19.6 percent of its third graders — met or exceeded expectations in literacy exams, up from 15.9 percent last year.

Students in Westminster also made strong improvements in literacy as the district posted a growth score of 55, surpassing the state average.

Westminster officials also highlighted gains for particular groups of students. Gaps in growth among students are narrowing.

Schools still on state ordered plans for improvement, and deadline for improvement

  • Bessemer Elementary, Pueblo, 2018
  • Heroes Middle, Pueblo, 2018
  • Risley International Academy, Pueblo, 2018
  • HOPE Online Elementary, Douglas 2019
  • HOPE Online Middle, Douglas, 2019
  • Prairie heights Middle, Greeley, 2019
  • Manaugh Elementary, Montezuma, 2019
  • Martinez Elementary, Greeley, 2019

Look up school results here.

One significant gap that narrowed in Westminster was between students who qualify for free or reduced-price lunch, a common measure of poverty, and those who don’t. In the math tests given to elementary and middle school students, the difference in growth scores between the two groups narrowed to three points from 10 points the year before, with scores hovering around 50.

Results in individual schools that are on state plans for improvement were more mixed. Three schools in Pueblo, for instance, all saw decreases in literacy growth, but increases in math. One middle school in Greeley, Prairie Heights Middle School, had significant gains in literacy growth.

The Aurora school district managed to get off the state’s watchlist last year, but one of its high schools is already on a state plan for improvement. Aurora Central High School has until 2019 to earn a higher state rating or face further state interventions.

Aurora Central High’s math gains on the SAT test exceeded last year’s, but improvement on the SAT’s literacy slowed. The school’s growth scores in both subjects still remain well below 50.

Look up high school test results here.