more tweaks

For third straight year, TNReady prompts Tennessee to adjust teacher evaluation formula

PHOTO: Grace Tatter
Education Commissioner Candice McQueen announced last April that she was suspending TNReady testing for grades 3-8 for the 2015-16 school year. Now, her department is asking lawmakers to make more adjustments to the weight of student test scores in Tennessee's teacher evaluation formula.

First, Tennessee asked lawmakers to make temporary changes to its teacher evaluations in anticipation of switching to a new test, called TNReady.

Then, TNReady’s online platform failed, and the state asked lawmakers to tweak the formula once more.

Now, the State Department of Education is asking for another change in response to last year’s test cancellation, which occurred shortly after the legislative session concluded.

Under a proposal scheduled for consideration next Monday by the full House, student growth from TNReady would count for only 10 percent of teachers’ evaluation scores and 20 percent next school year. That’s compared to the 35 to 50 percent, depending on the subject, that test scores counted in 2014-15 before the state switched to its more rigorous test.

The bill, carried by Rep. Eddie Smith of Knoxville, is meant to address teachers’ concerns about being evaluated by a brand new test.

Because testing was cancelled for grades 3-8 last spring, many students are taking the new test this year for the first time.

“If we didn’t have this phase-in … there wouldn’t be a relief period for teachers,” said Elizabeth Fiveash, assistant commissioner of policy. “We are trying to acknowledge that we’re moving to a new assessment and a new type of assessment.”

The proposal also mandates that TNReady scores count for only 10 percent of student grades this year, and for 15 to 25 percent by 2018-19.

The Tennessee Education Association has advocated to scrap student test scores from teacher evaluations altogether, but its lobbyist, Jim Wrye, told lawmakers on Tuesday that the organization appreciates slowing the process yet again.

“We think that limiting it to 10 percent this year is a wise policy,” he said.

To incorporate test scores into teacher evaluations, Tennessee uses TVAAS, a formula that’s supposed to show how much teachers contributed to individual student growth. TVAAS, which is short for the Tennessee Value-Added Assessment System, was designed to be based on three years of testing. Last year’s testing cancellation, though, means many teachers will be scored on only two years of data, a sore point for the TEA.

“Now we have a missing link in that data,” Wrye said. “We are very keenly interested in seeing what kind of TVAAS scores that are generated from this remarkable experience.”

Although TVAAS, in theory, measures a student’s growth, it really measures how a student does relative to his or her peers. The state examines how students who have scored at the same levels on prior assessments perform on the latest test. Students are expected to perform about as well on TNReady as their peers with comparable prior achievement in previous years. If they perform better, they will positively impact their teacher’s score.

Using test scores to measure teachers’ growth has been the source of other debates around evaluations.

Historically, teachers of non-tested subjects such as physical education or art have been graded in part by schoolwide test scores. The House recently passed a bill that would require the state to develop other ways to measure growth for those teachers, and it is now awaiting passage by the Senate.

 

deja vu

Last year, Ritz’s computer-based testing plan was largely dismissed. Today, McCormick adopted part of it as her own.

PHOTO: Shaina Cavazos
Glenda Ritz and Jennifer McCormick debated in Fort Wayne during the 2016 campaign this past fall.

Although she wasn’t on board with former-state Superintendent Glenda Ritz’s entire testing plan during last year’s campaign, current Indiana schools chief Jennifer McCormick today expressed support for a computer-based test format Ritz lobbied hard for during her last year in office.

These “computer-adaptive” exams adjust the difficulty-level of questions as kids get right or wrong answers. McCormick explained the format to lawmakers today when she testified on the “ILEARN” proposal that could replace the state’s unpopular ISTEP exam if it becomes law.

Computer-adaptive technology, she said, allows tests to be more tailored around the student. Test experts who spoke to Indiana policymakers this past summer have said the tests also generally take less time than “fixed-form” tests like the current ISTEP and could result in quicker turnaround of results.

During the summer, members of a state commission charged with figuring out what Indiana’s new testing system could look like largely argued against this testing format, including the bill’s author, Rep. Bob Behning, R-Indianapolis. At the time, he said he was concerned about investing in a technology-heavy plan when much of the state struggles to get reliable internet and computer access. Today, Behning didn’t speak against the concept.

Overall, McCormick was supportive of House Bill 1003, but she pointed out a few areas that she’d like to see altered. More than anything, she seemed adamant that Indiana get out of the test-writing business, which has caused Hoosiers years of ISTEP-related headaches.

Read: Getting rid of Indiana’s ISTEP test: What might come next and at what cost

“Indiana has had many years to prove we are not good test-builders,” McCormick told the Senate Education Committee today. “To continue down that path, I feel, is not very responsible.”

The proposed testing system comes primarily from the recommendations of the state commission. The biggest changes would be structural: The bill would have the test given in one block of time at the end of year rather than in the winter and spring. The state would go back to requiring end-of-course assessments in high school English, Algebra I and science.

The bill doesn’t spell out if the test must be Indiana-specific or off-the-shelf, and McCormick suggested the state buy questions from existing vendors for the computer-adaptive test for grades 3-8, which would have to be aligned with state standards.

For high school, McCormick reiterated her support for using the SAT and suggested making the proposal’s end-of-course assessments optional.

The ILEARN plan, if passed into law, would be given for the first time in 2019.

“Spring of 2019 is a more realistic timeline no matter how painful it is for all of us.” McCormick said. “We could do it for (2018), but it might not be pretty. We tried that before as a state, and we couldn’t get it right.”

You can find all of Chalkbeat’s testing coverage here.

Up and down

How changes to Michigan’s school ranking system hurt Cass Tech — and helped the DeVos family charter school

Detroit's selective Cass Technical High School saw its state ranking sink by 57 points in 2016 — one of dozens of Michigan schools that saw dramatic swings in their rankings after Michigan changed the tests kids take and the formula behind the rankings.

Some of Detroit’s most celebrated selective schools saw their standings plunge on the state’s most recent school rankings.

Renaissance High School was one of the highest ranked schools on Michigan’s 2014 Top to Bottom schools list, scoring in the 98th percentile, better than 98 percent of state schools. But when the state in January released its latest ranking, based on 2016 test scores, the school had dropped to the 48th percentile, putting it slightly below the state average.

Cass Technical High School dropped 57 points, from the 78th percentile in 2014 to the 21st percentile in 2016. (There was no 2015 list).

And the Bates Academy, a selective elementary school in northwest Detroit, dropped from the 86th percentile in 2014 to the 34th percentile last year.

The nosediving rankings could be alarming to parents and educators, but testing experts say the dramatic swings say more about a rating system that’s been in turmoil in recent years than it does about individual schools.

The state’s decision to change both the way it tests students and the way it translates student scores into a ranking means that dozens of schools saw their standings sink or soar by 50 or more points between 2014 and 2016 — far more movement than experts say can be explained by typical changes in schools from one year to the next.

Yet the rankings have created image problems for schools like Cass and Renaissance that saw their standings tank. They’ve made schools — like the Grand Rapids charter school founded by billionaire Dick DeVos and his wife, Betsy, the U.S. Education Secretary — look like they’ve made extraordinary improvements in just two short years. And they’ve raised questions about how officials can use the rankings to make crucial decisions such as which schools should be targeted for closure or intervention.

“It’s a very crude measure that’s being used to make a very important decision,” said Edward Roeber, who served as the state’s top testing official from 2003 to 2007.

The state’s plan to close as many as 38 schools based on the rankings is largely on hold for now as the affected districts negotiate improvement plans with the state, but the low-rated schools remain in danger of being closed next year.

And they’re not the only ones feeling the pain of the changing measures. Even higher-performing schools are trying to figure out where they stand this year and how they’ll fare next year when the state is expected to respond to a new federal law by scrapping the Top to Bottom list and replacing it with a new system.

“It’s difficult because the target keeps moving and there’s this really public document called the Top to Bottom list that’s out there for the world to see,” said Danielle Jackson, the chief academic officer for the University Prep charter school network.

When University Prep Math and Science High School saw its ranking drop 50 points from the 69th percentile in 2014 to the 19th percentile last year, the network reached out to parents to make sure they understood that the ranking formula had changed and that after years of preparing students for the ACT, kids were suddenly faced with a different test — the SAT — instead.

But those explanations only go so far in cities like Detroit where parents have many options and children can enroll in district, charter, private or suburban schools.

Here, a school that falls in the rankings can have a harder time recruiting students, potentially damaging its ability to survive.

“We’re in a highly competitive environment,” Jackson said.

With stakes that high, it’s important that schools have clear goals to work toward — and right now they don’t, said Sarah Lenhoff, a Wayne State University education professor who specializes in school improvement and choice.

“They’re sending really different and mixed signals, both to schools about what they need to work on to improve and to parents and families about what this ranking means,” Lenhoff said.

Lenhoff ran an analysis of the 2014 and 2016 rankings that identified 74 Michigan schools that saw their rankings go up or down by 50 or more points between 2014 and 2016. That includes 31 schools that fell precipitously in the rankings and 43 that leapt from the bottom to the top.

More than 500 schools saw a change of at least 25 points — roughly a fifth of the more than 2,500 schools that were ranked in both 2014 and 2016.

“You’ve got to wonder,” Lenhoff said “ Did those schools change that drastically or is there something going on where their ranking is not capturing the quality of the school in all dimensions?”

One of the schools that enjoyed a giant leap was the West Michigan Aviation Academy, the Grand Rapids charter school founded by the DeVos family.

That school went from the 32nd percentile in 2014 to the 87th percentile last year.

Does that mean it got better?

Maybe, or maybe not, said Sunil Joy, the assistant director of policy and research for Education Trust Midwest, a school advocacy organization.

“Michigan has by far one of the most complex accountability systems in the country and that makes it really difficult for the public and educators and schools to really understand what’s behind the calculation,” Joy said. “With such an overly complex system, you can’t really pinpoint what happened.”

State officials say they know that their rating system has been mercurial.

Not only have the exams behind the the ratings changed from the MEAP to the M-STEP in elementary and middle school and from the ACT to the SAT in high school, but the state also made major changes to the formula it uses to calculate rankings.

The biggest change to the formula was the state’s decision not to factor a school’s so-called achievement gap into its final score in 2016.

The achievement gap, which measures the difference between the highest-performing and lowest-performing students in a school, accounted for 25 percent of a school’s ranking in 2014 but wasn’t part of the 2016 ranking because officials feared that gap scores had been artificially inflating the rankings of low-achieving schools where nearly all students posted low test scores.

The state also changed the way it measures whether students improved from one year to the next.

So if a school dropped in the rankings, it could be because students have not adapted well to the new exams. Or the school could have lost points to the new formula.

“Unfortunately, we’ve been so busy with (responding to the new federal education law), we haven’t really had a chance to look into the old data,” said Chris Janzer, who heads the school accountability office at the state education department.

The principals of Cass Tech, Renaissance and Bates did not respond to requests for comment about the schools’ drop in the rankings. A spokeswoman for the Detroit Public Schools Community District declined to comment.

Janzer said recent changes to the ranking system were intended to be the last major tweaks for a while. But a new federal law that passed in 2015 is expected to force another big change. The state has for months been discussing a shift to a letter grade rating system but the AP reported Monday that letter grades are off and a school report card could be in.

Critics of the frequent changes make “a valid point,” Janzer said. “When we’re charged with designing a new system, we push for a lengthy life span for it.”

But the education department has limited control at a time when state lawmakers, partisan politics and federal law have all had a hand in altering the way Michigan students and schools have been judged in recent years.

The Education Department told schools and the federal government that there would be no high-stakes consequences for test scores in 2015 and 2016 because schools needed time to adapt to new exams and a new rating system.

But a different state office, the School Reform Office, which Gov. Snyder moved out of the Education Department in 2015 so it would report to him, announced last summer that it wasn’t held to the department’s commitments. The reform office said it was obliged to follow a new law requiring the state to shut down every Detroit school that had been in the bottom five percent of state rankings for three years in a row. The office said it would apply the mandate to the entire state.

The education department did not release its Top to Bottom list in 2015, but the School Reform Office put out a limited list last year identifying schools that were in the bottom five percent in 2015.

When the full 2016 Top to Bottom list came out in January, the reform office announced that 38 schools that appeared at the bottom of the 2014, 2015 and 2016 lists were in danger of closing.

The closings have been postponed for at least 18 months but it’s not clear what will happen to those schools next year — or to the 35 schools that were put on notice that they could be closed in 2018 if students don’t do well on this year’s exams.

It can be a tough environment in which to teach kids, Jackson said, but she says she tries to tune out the noise.

“This is the reality,” she said. “I don’t have the luxury to kind of roll up in a ball on the floor and cry. I don’t have the luxury to get on the soap box and talk about ‘This isn’t fair.’ My job is to state the facts to my team and to be able to respond in the most responsible way without making the school a place where kids only come to be drilled on tests.”

Still, Jackson called on the state to settle on one rating and “hold this target steady.”

“I surely hope that we can be really, really clear,” she said. “The most underserved students in our community deserve an opportunity to be successful and stability is a very, very big part of making that happen.”

For the full list of Michigan school rankings in 2014 and 2016, click here.