why the jump?

What caused New York City’s state test scores to jump?

PHOTO: Monica Disare
State Education Commissioner MaryEllen Elia visited the School of Diplomacy in the Bronx in 2015.

When State Commissioner MaryEllen Elia announced this year’s state test scores, she said she wasn’t sure exactly what caused such a big statewide bump — nearly 7 percent in English proficiency and 1 percent point in math.

“We cannot pinpoint exactly why the test [scores] increased,” she told reporters on Friday afternoon.

Her comments immediately turned the spike in scores into an education-world Rorschach test, and everyone saw something different in the inkblot. Mayor Bill de Blasio immediately claimed victory for the city’s almost 8 percent increase in English proficiency, while charter school advocates zeroed in on the even bigger increase in charter test scores, and researchers rolled their eyes, pointing out that test scores are an unreliable marker of progress — especially when the tests themselves have changed.

So who’s right? The answer likely involves some combination of student learning and test tweaks. We’ve compiled a list of the most prominent theories and looked at the evidence for each.

The de Blasio reforms are working

City officials wasted no time claiming de Blasio-era reforms drove the rise in test scores.

“A lot is changing, and this is pure, hard evidence that these changes are working, and we expect a lot more to come,” said de Blasio at a Monday press conference. He cited his “Renewal” program for struggling schools; his administration’s support of community schools, which offer additional services to families; and his universal pre-K push.

De Blasio’s case is supported by the fact that city proficiency rates increased more, on average, than test scores statewide. While the percentage of students passing state English scores increased by 6.6 percent, the city’s increased by 7.6 percent. Commissioner Elia also gave the city kudos, saying a renewed focus on teacher training and writing might explain the jump in scores.

State tests got easier

Could de Blasio-era reforms explain the entire increase in test scores? Probably not.

State tests across the state went up significantly — so much that Elia herself cautioned this year’s test scores are not an “apples-to-apples” comparison to last year’s. In response to the backlash over the introduction of Common Core-based assessments, officials made a number of changes to the tests this year, including shortening them and giving students unlimited time. Researchers said those changes likely explain some, if not much, of the statewide increase.

The increases “are sufficiently large that it makes me think there’s something about the difference in the tests from last year that accounts for the difference in growth,” said Aaron Pallas, a professor of sociology and education at Teachers College at Columbia University.

Charter schools are part of the answer

Just as quickly as Fariña and de Blasio celebrated the rise in scores, charter school advocates — frequent rivals of de Blasio — jumped in with their own good news.

City charter school English proficiency rate went up by 13.7 percent, beating the city’s overall average increase by a fair margin. Success Academy CEO Eva Moskowitz dismissed the rising scores at traditional district schools since they mirrored the state’s more closely and could thus be explained by the test changes, she argued. To “find real improvement,” she wrote in the New York Daily News, officials should look to charter schools instead.

New York City charter schools’ scores are analyzed separately from district schools, and so the charter growth didn’t contribute to — or account for — the city’s bump, state officials said. But their scores did contribute to the statewide increase.

The Common Core is working

There might be other explanations, but here’s the last one we’ll explore: The Common Core is working.

In 2013, state officials implemented tests aligned to the more rigorous Common Core learning standards. Experts knew the new tests would likely cause an immediate drop in scores, but officials hoped that over time, students and teachers would adjust to the new material and eventually test scores would rise.

Could this be a sign they were right? One piece of evidence to support that theory is the fact that the biggest increases in English proficiency were among third-graders, who started their elementary school education with a Common Core curriculum. Third grade proficiency levels in the state increased by 10.9 percent.

That did not go unnoticed by the Education Trust, a nonprofit that heralded the progress on state tests as a sign that higher standards work.

“The Common Core state standards and tests have been unfairly demonized and used to excuse the failures of our education system,” two leaders of the group wrote. “When we truly listen to what teachers, parents and students are saying, we know that high standards, implemented well, enable students to thrive.”

In the end, it’s likely too early to know exactly what drove the results, said Pallas, the Columbia testing expert. He is trying to isolate how much of the change has to do with test structure, as opposed to better instruction or learning. Right now, he said, parsing the two is tricky.

“There’s just too many moving parts right now,” he explained. “We’ll be able to have a better sense of what’s going on [eventually], but right now we’re in this gray area.”

 

not so fast

Why Tennessee legislators share blame, too, for TNReady testing headaches

PHOTO: TN.gov
Tennessee lawmakers listen to Gov. Bill Haslam deliver his 2016 State of the State address at the State Capitol in Nashville.

Exasperated with another round of testing problems in Tennessee public schools, state lawmakers have used their bully pulpit to rail against Education Commissioner Candice McQueen for her management of the state’s beleaguered standardized test.

Last week, they called her in to the State Capitol for a two-hour grilling about online snafus and a reported cyber attack that got TNReady testing off to another rocky start. Several called for McQueen’s resignation.

The next day, lawmakers dramatically stepped in and passed legislation so that this year’s scores mostly won’t count on student grades or in important decisions about teachers and schools, essentially gutting the state’s vaunted accountability system, at least for this year.

Few legislators have been willing to talk about the elephant in the room, but several education advocates have.

Just four years ago, PARCC was to be the vehicle for Tennessee students to begin testing online using questions aligned to Common Core academic standards for math and English language arts. At the time, Tennessee was a Common Core state and had been working for several years toward sharing an online test through a multi-state consortia known as PARCC, short for the Partnership for Assessment in College and Career Readiness.

But in April 2014, six months before testing was supposed to begin and amid political backlash over Common Core, the legislature voted to pull out abruptly from the partnership.

The decision, which was against the wishes of Gov. Bill Haslam and former Education Commissioner Kevin Huffman, set the state’s collision course toward TNReady. It meant that Tennessee had to develop a new test — pronto — and find a new company to administer it. Measurement  Inc., a small North Carolina-based firm, was hired for $108 million in November 2014.

Generally, it takes at least two years to create a test and prep for launch. State lawmakers gave Measurement Inc. about a year, with the first testing starting in the fall of 2015 for some high school students. But the real test came in February 2016. That’s when most students in grades 3-11 logged on and the platform collapsed on the very first day, the victim of too many students trying to test at one time with too few computer servers.

McQueen, who had replaced Huffman after the deal was inked with Measurement Inc., subsequently scrubbed tests for most students that year and fired the state’s testing company. A few months later, she turned to Minnesota-based Questar to pick up the pieces for $30 million annually beginning in July of 2016. Things went slightly better the next year with TNReady, though not perfectly.

This year, a lot was riding on Tennessee officials to get TNReady right in a return to statewide online testing. But when more technical problems erupted on the first day this spring, McQueen immediately became the target for blame.

PHOTO: TN.gov
Tennessee Education Commissioner Candice McQueen has been under fire for her oversight of the state’s standardized test, known as TNReady, which has had a string of high-profile problems since its 2016 rollout.

“We do want to have that one throat to choke,” she told reporters who asked who should be held accountable, before adding that “there’s lots and lots of people involved.”

During a legislative hearing that same day, Rep. Mark White reminded his colleagues about their pivotal 2014 decision to pull out of the testing consortia.

“General Assembly, we had some ownership in this,” said the Memphis Republican, who also voted to exit the partnership. “We had a testing company originally four years ago …[but] we pushed back against the commissioner and the Department of Education and said we don’t want PARCC for political reasons. … We fussed about Common Core and we fussed about the standards.”

Tennessee wasn’t alone. In 2014, it was one of 18 states that comprised the consortia. The partnership is now down to four states — Illinois, Maryland, New Jersey, and New Mexico — along with Washington, D.C., and schools operated by the U.S. Department of Defense and the Bureau of Indian Education.

"General Assembly, we had some ownership in this."Rep. Mark White, R-Memphis

The exodus was due to mostly Republican complaints of federal overreach by the Obama administration for incentivizing states to adopt the controversial Common Core standards. But superintendents back home were also fearful of the switch to computerized testing.

Online testing has gone fairly smoothly for those that stayed in the partnership, and PARCC is now the only assessment fully approved by the U.S. Department of Education.

“States that have continued with the program now have four years of longitudinal data measuring student performance and growth over time,” said Arthur VanderVeen, who leads New Meridian, the company that manages the partnership.

The shared test also has been a money saver.

Hanna Skandera, New Mexico’s former secretary of education, said economies of scale allowed her state to cut testing costs by more than a quarter by sticking with PARCC. More importantly, she said, the test has been an effective measuring stick.

“This is not about a brand. It’s about quality of assessment,” said Skandera, who formerly chaired the partnership’s board. “PARCC allowed us to measure the things we cared about — critical thinking, higher expectations aligned to higher education. In New Mexico, it’s been a big step in the right direction.”

Below, you can view a timeline of Tennessee’s testing journey from PARCC to TNReady.

new rules

Now that TNReady scores will count less for students, will they even try?

PHOTO: Nicholas Garcia

In the face of a statewide testing debacle, the Tennessee legislature’s hasty edict this week to discount test results has mollified some teachers and parents, but raised more questions about the role of test scores and further eroded the motivation of students, who must labor for about two more weeks on the much-maligned TNReady test.

Thursday’s sweeping measure to allow districts to ignore test results when grading students and to prohibit the use of test scores when determining teacher compensation has left educators and students shrugging their shoulders.

“I’ve gone from ‘oh well, tests are just a part of life’ to ‘this is an egregious waste of time and resources and does not respect the developmental needs of our children,’” said Shelby County parent Tracy O’Connor. For her four children, the testing chaos has “given them the idea that their school system is not particularly competent and the whole thing is a big joke.”

Her son, Alex O’Connor, was even more succinct. “We spend $30 million on tests that don’t work, but we can’t get new textbooks every year?” said the 10th-grader at Central High School. “What’s up with that? I’m sure half of us here could design a better test. It’s like buying a used car for the price of a Lamborghini.”

The legislature’s decision created a new challenge for Tennessee’s Department of Education, which planned to use 2018 TNReady testing data to rate and identify the lowest-performing schools, as required by the federal government. Now, with the test’s reliability under question, state officials say they are determining “additional guidance” to provide districts on how the state will comply with the U.S. Department of Education.

Student test results still will be used to generate a score for each teacher in the Tennessee Value-Added Assessment System, known as TVAAS. Scores will count for 20 percent of teachers’ evaluations, though districts now cannot use the scores for any decisions related to hiring, firing, or compensating teachers.

For students, local school boards will determine how much TNReady scores will count toward final grades — but only up to 15 percent. Several school districts have already expressed serious reservations about the testing data and likely won’t use them in students grades at all. And in previous years, the results didn’t come back in time for districts to incorporate them anyway.

In sum, asked Memphis sophomore Lou Davis, “Why are we doing this anymore when know it won’t count?”

About 650,000 students are supposed to take TNReady this year, with 300,000 of them testing online, according to the state. Each student takes multiple tests. As of Friday, more than  500,000 online tests sessions had been completed.

Even as testing continues, some education leaders worry the exam’s credibility is likely to sink even further, because students might not try, and parents and teachers may not encourage much effort.

“In the immediate term, there’s concern about how seriously people will take the test if they know it’s not going to count,” said Gini Pupo-Walker, head of the Tennessee Educational Equity Coalition and a member of the state’s testing task force. “Will students continue to take the test? Will kids show up? Will parents send their kids to school?” she asked. “Now, there’s the whole question of validity.”

Sara Gast, spokeswoman for the Department of Education, said while the new legislation provides more flexibility for districts in how they use TNReady results, it doesn’t mean that the results don’t matter.

“The results always matter. They provide key feedback on how students are growing and what they are learning, and they provide a big-picture check on how well they are mastering our state academic expectations,” Gast said. “It serves as accountability for the millions of taxpayer dollars that are invested into public education each year.”

Jessica Fogarty, a Tullahoma school board member and parent, says she thinks this year’s testing issues could lead to more parents telling their kids to refuse state tests in the future.

A proponent of opting out of state tests, Fogarty said, “We need to understand that we can choose what our children do or do not suffer through. I hope this debacle showed parents what a waste of time this is — students would gain more through reading a book.”

Because Tennessee has no official opt-out policy, students wanting to opt out must “refuse the test” when their teacher hands it to them.

Jennifer Proseus, a parent of a student at Bartlett High School, said her daughter has opted out of state testing in the past, but started taking the exams this year because she believed it could affect her final grades.

“With college looming in a couple years, she couldn’t afford to get zeroes on her report cards,” Proseus said. But with the test debacle, her daughter might change her mind and just skip the remaining two weeks of testing.

“I even took the online practice TNReady a few years ago and it was terribly confusing to navigate,” Proseus said. “The testing in Tennessee is not transparent — it is almost like it is set up to trick and fail children — and that’s very cruel for a young child to deal with.”