testing 1-2-3

If state tests keep changing, should they still be used to judge struggling schools?

PHOTO: Monica Disare
Chancellor Carmen Fariña

In a packed room of educators, New York City schools Chancellor Carmen Fariña announced that the city’s turnaround program for struggling schools is making extraordinary progress.

“I want to be clear,” Fariña said. “English proficiency … increased at 59 out of the 63 [Renewal] schools. Let me say this again, 59 out of 63 schools.”

Fariña is correct, but the state offered a more tempered assessment of the scores. On Friday, State Education Commissioner MaryEllen Elia said changes to this year’s test, such as offering students unlimited time and asking fewer questions, meant last year’s scores were not an “apples-to-apples comparison” with last year’s.

Her statement underscores what critics see as a dilemma: As tests continue to change, how can officials judge yearly progress on major initiatives?

That question is particularly relevant to two high-profile programs for struggling schools — the state’s receivership program and the city’s “Renewal” school program — both of which carry penalties for underperforming schools and use test scores as one way to gauge student progress. Renewal schools are expected to show improvements between 2015 and 2017, but the test process has changed within that timeframe — and could change again next year.

“It’s like trying to judge the success of a weight loss program when you have three different scales that you can’t count on,” said Aaron Pallas, a professor of sociology and education at Teachers College at Columbia University.

City officials defended the comparison between 2015 and 2016 test results, saying the rigor of the exam remained the same this year, only the structure of the test changed. They also noted that multiple benchmarks will be used to evaluate “Renewal” schools, not just test scores.

“These tests are not easier,” Fariña said during a Monday press conference. “I want to be clear on that. These tests had the same rigor as the one they took last year.”

State officials reiterated that the tests were “comparably rigorous” to last year’s assessments and said they will review the indicators used to judge improvement in struggling schools and make sure they are “working as intended.”

Even if the comparison is flawed, some say, the test scores are still useful in assessing progress. “We have to have some point of comparison for how our students are doing, as imperfect as it might be,” said David Albert, spokesman for the New York State School Boards Association.

This year is not the first in which tests have changed — far from it. The tests have been revised multiple times over the last decade, with certain years showing large swings due to those changes.

After test scores dropped in 2013 with the introduction of Common Core standards and exams, the state vowed to revise the test to address concerns.

That process will likely take years, and during the transition period, grades 3-8 math and English tests will not be used to evaluate teachers. But state and city officials are still using those tests to judge struggling schools, and that’s a problem, said David Bloomfield, an education professor at Brooklyn College and the CUNY Graduate Center. Schools on the state’s or city’s lists of low-performing schools could face consequences, such as being taken over by an outside receiver or closed, if they fail to meet academic benchmarks.

“This isn’t a new phenomenon, it’s just that earlier there weren’t high stakes,” Bloomfield said. “Now because of the particularly short time span that’s involved, they are being used way beyond their ability for accurate measurement.”

Even without changes to the test, yearly fluctuations should be viewed carefully, said Roey Ahram, director of research and evaluation at the NYU Steinhardt Metropolitan Center for Research on Equity and the Transformation of Schools.

“You always have to look at test scores with a grain of salt, whether they are changing or not,” he said. “The asterisk is there for a reason.”

Not Ready

Memphis students won’t see TNReady scores reflected in their final report cards

PHOTO: Creative Commons / timlewisnm

Shelby County Schools has joined the growing list of Tennessee districts that won’t factor preliminary state test scores into students’ final grades this year.

The state’s largest school district didn’t receive raw score data in time, a district spokeswoman said Tuesday.

The State Department of Education began sharing the preliminary scores this week, too late in the school year for many districts letting out in the same week. That includes Shelby County Schools, which dismisses students on Friday.

While a state spokeswoman said the timelines are “on track,” Superintendent Dorsey Hopson said the timing was unfortunate.

“There’s a lot of discussion about too many tests, and I think anytime you have a situation where you advertise the tests are going to be used for one thing and then we don’t get the data back, it becomes frustrating for students and families. But that’s not in our control,” he said Tuesday night.

Hopson added that the preliminary scores will still get used eventually, but just not in students’ final grades. “As we get the data and as we think about our strategy, we’ll just make adjustments and try to use them appropriately,” he said.

The decision means that all four of Tennessee’s urban districts in Memphis, Nashville, Knoxville and Chattanooga won’t include TNReady in all of their students’ final grades. Other school systems, such as in Williamson and Wilson counties, plan to make allowances by issuing report cards late, and Knox County will do the same for its high school students.

Under a 2015 state law, districts can leave out standardized test scores if the information doesn’t arrive five instructional days before the end of the school year. This year, TNReady is supposed to count for 10 percent of final grades.

Also known as “quick scores,” the data is different from the final test scores that will be part of teachers’ evaluation scores. The state expects to release final scores for high schoolers in July and for grades 3-8 in the fall.

The Department of Education has been working with testing company Questar to gather and score TNReady since the state’s testing window ended on May 5. About 600,000 students took the assessment statewide in grades 3-11.

State officials could not provide a district-by-district listing of when districts will receive their scores.

“Scores will continue to come out on a rolling basis, with new data released every day, and districts will receive scores based on their timely return of testing materials and their completion of the data entry process,” spokeswoman Sara Gast told Chalkbeat on Monday. “Based on district feedback, we have prioritized returning end-of-course data to districts first.”

Caroline Bauman and Laura Faith Kebede contributed to this report.

Making the grade

TNReady scores are about to go out to Tennessee districts, but not all will make student report cards

PHOTO: Chalkbeat Photo Illustration

The State Department of Education will start Monday to distribute the test score data that goes into students’ final report cards, but it won’t arrive in time for every district across the state.

That’s because some districts already have ended their school years, some won’t have time to incorporate TNReady grades before dismissing their students, and some missed the state’s first deadline for turning in testing materials.

“Our timelines for sharing TNReady scores are on track,” spokeswoman Sara Gast said Friday, noting that the schedule was announced last fall. “We have said publicly that districts will receive raw score data back in late May.”

Shelby County Schools is waiting to see when their scores arrive before making a decision. A spokeswoman said Tennessee’s largest district met all testing deadlines, and needs the scores by Monday to tabulate them into final grades. The district’s last day of school is next Friday.

School leaders in Nashville and Kingsport already have chosen to exclude the data from final grades, while Williamson County Schools is delaying their report cards.

A 2015 state law lets districts opt to exclude the data if scores aren’t received at least five instructional days before the end of the school year.

TNReady scores are supposed to count for 10 percent of this year’s final grades. As part of the transition to TNReady, the weight gradually will rise to between 15 and 25 percent (districts have flexibility) as students and teachers become more familiar with the new test.

The first wave of scores are being sent just weeks after Education Commissioner Candice McQueen declared this year’s testing a “success,” both on paper and online for the 24 districts that opted to test high school students online this year. Last year, Tennessee had a string of TNReady challenges in the test’s inaugural year. After the online platform failed and numerous delivery delays of printed testing materials, McQueen canceled testing in grades 3-8 and fired its previous test maker, Measurement Inc.

Tennessee test scores have been tied to student grades since 2011, but this is the first year that the state used a three-week testing window instead of two. Gast said the added time was to give districts more flexibility to administer their tests. But even with the added week, this year’s timeline was consistent with past years, she said.

Once testing ended on May 5, school districts had five days to meet the first deadline, which was on May 10, to return those materials over to Questar, the state’s new Minneapolis-based testing company.

School officials in Nashville said that wasn’t enough time.

“Due to the volume of test documents and test booklets that we have to account for and process before return for scoring, our materials could not be picked up before May 12,” the district said in a statement on Thursday.

Because districts turned in their testing materials at different times, the release of raw scores, will also be staggered across the next three weeks, Gast said.