Are Children Learning

Indiana rank continues to rise on the national NAEP test

PHOTO: Shannan Muskopf via Flickr

Indiana did as good or better than the U.S. average on the National Assessment of Educational Progress, or the “nation’s report card,” this year compared to 2013, while many states saw significant drops.

For 2015, the state was ranked fourth in the nation in fourth-grade math and 11th for eighth-grade math, up from a ranking of fourth and 19th, respectively, in 2013. In reading, Indiana students ranked 10th for fourth-grade reading and 16th for eighth-grade reading, up from 15th and 27th. Back in 2013, the state made huge jumps in scores that some questioned the validity of at the time, said Mark O’Malley, the state’s NAEP coordinator.

“That was a big deal and that made the headlines,” O’Malley said. “As Indiana repeats again the fourth highest math score in fourth grade, it kind of validates the student performance that has happened, so that is exciting. It’s not a phenomenon, it didn’t happen by chance.”

 

Indiana students scoring at or above proficiency on NAEP math

 

Indiana students scoring at or above proficiency on NAEP reading

Data source: NAEP and National Center for Education Statistics, Graphics by: Sarah Glen/Chalkbeat

None of the change in this year’s score have “statistical significance” from 2013, meaning they don’t necessarily indicate student achievement is any lower or higher this year than last time — likely the change is due to chance, O’Malley said.

However he said that’s not necessarily a bad thing given the state’s more challenging academic standards and big changes to state tests that can cause some chaos in the classroom.

“Indiana held on and plateaued,” O’Malley said. “The state assessment is up and down, and in the classroom it’s just so chaotic and frustrating. The students were able to hold on to their student performance in the NAEP assessment, and all the other states around us, their scores went down dramatically.”

Compared to 2000, Indiana students have measurably improved in fourth-grade math and reading and in eighth-grade math over 15 years. Eighth-grade reading scores aren’t significantly different from 2000.

Massachusetts, Minnesota and New Hampshire outranked Indiana in fourth-grade math scores, yet the differences are very small. Indiana English-learners achieved scores 16 points higher than the national average among other students learning English as a new language.

In general, Indiana saw strong performance from English-learners and students with special needs. All states are required by the federal government to take NAEP, which is given to sample of students every other year. About 12,000 students took the test in Indiana, O’Malley said.

In some states, lower scores might be affected by having more students with special needs and English-learners taking the test, but Arne Duncan, the U.S. Secretary of Education, said that shouldn’t be considered a problem.

“This is not a problem or a deficit, it’s a huge opportunity to change the lives of young people through educational opportunities,” Duncan said. “We should be proud schools are becoming more diverse, and we should be proud to hold ourselves accountable for the education of all students.”

Nationwide, fourth- and eighth-graders posted slightly lower scores on the math and reading tests than in 2013, the last time the tests were administered.

“We’re trying not to read too much in a decline in at this point,” said Peggy Carr, acting commission of the National Center of Education Statistics, which administers NAEP. “We understand it’s a pattern consistent across many states, but … we don’t know yet if these changes we’re talking about today are long term.”

Scores in both subjects have risen significantly over time, suggesting that the average American student’s skills have improved. Math scores have risen by 27 points out of 500 and reading scores have risen by 10 points since the first time the exams were given, but achievement gaps between black and white students have not narrowed substantially.

U.S. students have taken the NAEP exams for decades in an effort to provide a consistent measure of student performance at a time when states’ standards varied widely.

Now, many states are in the process of adopting new tests that reflect shared standards, potentially allowing for more detailed and frequent comparisons of students across the country. That means for some states, NAEP now serves the dual role of testing student performance across states and assessing whether states have achieved their goal of crafting more “accurate” measures of student achievement with their new exams.

Yet because Indiana abandoned Common Core for state-specific standards in 2014, such comparisons between Indiana’s standards and those in other states aren’t really possible. That makes NAEP even more valuable to educators and policymakers to keep tabs on how students are doing compared to the rest of the country, O’Malley said.

“This is one thing that every state takes in the same format, it’s administered the same,” he said. “It’s the only thing left that … can measure student achievement universally.”

new rules

Now that TNReady scores will count less for students, will they even try?

PHOTO: Nicholas Garcia

In the face of a statewide testing debacle, the Tennessee legislature’s hasty edict this week to discount test results has mollified some teachers and parents, but raised more questions about the role of test scores and further eroded the motivation of students, who must labor for about two more weeks on the much-maligned TNReady test.

Thursday’s sweeping measure to allow districts to ignore test results when grading students and to prohibit the use of test scores when determining teacher compensation has left educators and students shrugging their shoulders.

“I’ve gone from ‘oh well, tests are just a part of life’ to ‘this is an egregious waste of time and resources and does not respect the developmental needs of our children,’” said Shelby County parent Tracy O’Connor. For her four children, the testing chaos has “given them the idea that their school system is not particularly competent and the whole thing is a big joke.”

Her son, Alex O’Connor, was even more succinct. “We spend $30 million on tests that don’t work, but we can’t get new textbooks every year?” said the 10th-grader at Central High School. “What’s up with that? I’m sure half of us here could design a better test. It’s like buying a used car for the price of a Lamborghini.”

The legislature’s decision created a new challenge for Tennessee’s Department of Education, which planned to use 2018 TNReady testing data to rate and identify the lowest-performing schools, as required by the federal government. Now, with the test’s reliability under question, state officials say they are determining “additional guidance” to provide districts on how the state will comply with the U.S. Department of Education.

Student test results still will be used to generate a score for each teacher in the Tennessee Value-Added Assessment System, known as TVAAS. Scores will count for 20 percent of teachers’ evaluations, though districts now cannot use the scores for any decisions related to hiring, firing, or compensating teachers.

For students, local school boards will determine how much TNReady scores will count toward final grades — but only up to 15 percent. Several school districts have already expressed serious reservations about the testing data and likely won’t use them in students grades at all. And in previous years, the results didn’t come back in time for districts to incorporate them anyway.

In sum, asked Memphis sophomore Lou Davis, “Why are we doing this anymore when know it won’t count?”

About 650,000 students are supposed to take TNReady this year, with 300,000 of them testing online, according to the state. Each student takes multiple tests. As of Friday, more than  500,000 online tests sessions had been completed.

Even as testing continues, some education leaders worry the exam’s credibility is likely to sink even further, because students might not try, and parents and teachers may not encourage much effort.

“In the immediate term, there’s concern about how seriously people will take the test if they know it’s not going to count,” said Gini Pupo-Walker, head of the Tennessee Educational Equity Coalition and a member of the state’s testing task force. “Will students continue to take the test? Will kids show up? Will parents send their kids to school?” she asked. “Now, there’s the whole question of validity.”

Sara Gast, spokeswoman for the Department of Education, said while the new legislation provides more flexibility for districts in how they use TNReady results, it doesn’t mean that the results don’t matter.

“The results always matter. They provide key feedback on how students are growing and what they are learning, and they provide a big-picture check on how well they are mastering our state academic expectations,” Gast said. “It serves as accountability for the millions of taxpayer dollars that are invested into public education each year.”

Jessica Fogarty, a Tullahoma school board member and parent, says she thinks this year’s testing issues could lead to more parents telling their kids to refuse state tests in the future.

A proponent of opting out of state tests, Fogarty said, “We need to understand that we can choose what our children do or do not suffer through. I hope this debacle showed parents what a waste of time this is — students would gain more through reading a book.”

Because Tennessee has no official opt-out policy, students wanting to opt out must “refuse the test” when their teacher hands it to them.

Jennifer Proseus, a parent of a student at Bartlett High School, said her daughter has opted out of state testing in the past, but started taking the exams this year because she believed it could affect her final grades.

“With college looming in a couple years, she couldn’t afford to get zeroes on her report cards,” Proseus said. But with the test debacle, her daughter might change her mind and just skip the remaining two weeks of testing.

“I even took the online practice TNReady a few years ago and it was terribly confusing to navigate,” Proseus said. “The testing in Tennessee is not transparent — it is almost like it is set up to trick and fail children — and that’s very cruel for a young child to deal with.”

Chalkbeat explains

Four reasons Tennessee likely won’t go back to paper testing

As another wave of online testing problems plagues Tennessee schools, one of the solutions proposed by state legislators — go back to paper exams — is a stretch for a state that has invested millions into electronic exams.

In short, reverting to pencil-and-paper would be akin to ordering iPhone users to go back to flip phones. It almost certainly won’t happen.

Two Memphis-area state lawmakers want to ban the online version of TNReady starting next school year until the state comptroller determines its problems are “fully and completely fixed.” Another lawmaker says local districts should be able to choose between paper and electronic testing.

Their motivation springs from this week’s technical problems with TNReady, just two years after a statewide online collapse prompted Education Commissioner Candice McQueen to cancel most state testing that year.

Despite the frustrations, here are four reasons why it’s unlikely Tennessee will go back to paper testing:

Superintendents think they’ve gone too far to turn back now. Maryville Director of Schools Mike Winstead cautioned against rash decisions in the heat of the moment.

“When things like this happen, it’s easy to overreact,” he told Chalkbeat. “But we’ve come too far. We know that online testing is the future. If we turn back, it will take a long time to get back to where we were.”

School systems and counties have poured millions into infrastructure and devices, added Dale Lynch, executive director of the Tennessee Organization of School Superintendents.

“We don’t want to back up. We want to get it right, though,” he said.

Paper is more time consuming. With online testing, “we can get test materials [and scores] back or to folks much quicker,” McQueen said this week.

Preparing paper tests requires hours of sorting and labeling exams. And if the materials arrive late, like they did for several districts this month because of severe weather at testing company Questar’s printing center in the Northeast, the time crunch is especially stressful.

Granted, a top-notch online system that protects against cheating and hacking could be more expensive than a paper version, said Wayne Camara, an ACT research chair who oversees test security.

“The issue of cost is relative.” he said. Multiple versions of computer tests are necessary to help safeguard against cheating, especially via social media.

“If you’re having to produce 10 or 15 forms of a computer test, most likely it’s not cheaper.”

If Tennessee switches back to paper testing, it will be one of few states nationwide. A recent analysis by John Hopkins School of Education lists 11 states that were still using paper tests in 2016 for elementary students. For middle schools, it was nine states.

Nearly across the board, those states with no experience with online testing did worse in national online testing.


Read more about Tennessee’s most recent performance on the Nation’s Report Card.


There’s security issues with paper, too. The alleged cyber attack on Questar’s data center Tuesday sparked a statewide outcry, but switching back to paper won’t eliminate security issues.

“Both digital- and paper-based testing are certainly susceptible to cheating,” said Camara, the testing cybersecurity expert. “I don’t think anybody would say that there’s a significant reduction of security measures or cheating with computers. It’s just different.”

One of the largest state test cheating scandals happened in Atlanta with paper tests when principals and teachers changed student answers. That’s much harder to do online.

Jacinthia Jones and Marta W. Aldrich contributed to this story.