high stakes

There’s always been confusion surrounding Tennessee’s growth model. With a missing year of data, new questions pile on

At a time when scores are about to be used for high-stakes decisions in how to improve Tennessee’s schools, gaps in the state’s data and uncertainty about how scores were derived have left Memphis officials wondering how to interpret the torrent of information.

Last year’s chaotic state testing, which led to the cancellation of the state’s test for grades 3 to 8, left a crucial gap in the data meant to help make decisions about schools and teachers.

School leaders have also said they were puzzled by the state’s methodology in reaching the so-called growth scores upon which districts and schools are judged — particularly by how they arrived at the Memphis district’s low score.

Even those who are paid to sift through the data say they are having trouble getting answers to questions about the growth scores, known as TVAAS. Bill White, chief of planning and accountability for Shelby County Schools, conceded to board members last week that he didn’t know the ins and outs of the complex formula and the changes meant to compensate for the missing data.

“I have personally never been shown all the mathematics behind our data and how this works,” he told board members. “I do know that it has been peer-reviewed and vetted and it’s essentially been held up among those statisticians. But there is a lot that goes on behind the scenes that no one has been able to walk us through.”

The confusion has renewed skepticism about the state’s value-added model, which is supposed to help officials identify the impact that schools and teachers have on student performance. The system relies on the state’s data measuring student growth in districts.

Part of the problem is last year’s botched testing, which is having multiple ripple effects throughout the state.

This year, growth scores are comparing 2016-17 test results with the 2014-15 school year, the most recent data available. That throws a wrench in how to assess which school or teacher is responsible for a child’s growth over a two-year period. And for elementary schools, that means there is no data for fourth graders this year since testing in third grade, the first year students take state tests, was canceled.

In addition, one subject was dropped entirely from TVAAS calculations because social studies questions were a trial run for elementary and middle schools students and did not count.

Statisticians for the most part have figured out how to calculate growth even when a state transitions to a new test. But the missing data creates a whole other host of challenges the revisions attempt to account for.

One Memphis charter leader said he still isn’t quite sure how his school even got a score since last year his highest grade level at the school was third grade, the first year of testing.

“It’s such a convoluted formula, it’s hard for us to understand. We’re not sure how we got (our score),” said the charter leader, who declined to be named because he was still seeking answers from the state.

Damian Betebenner, a senior associate at Center for Assessment that regularly consults with state departments, said missing data on top of a testing transition “muddies the water” on results.

“When you look at growth over two years, so how much the student grew from third to fifth grade, then it’s probably going to be a meaningful quantity,” he said. “But to then assert that it isolates the school contribution becomes a pretty tenuous assertion… It adds another thing that’s changing underneath the scene.”

At the same time, TVAAS scores for struggling schools will be a significant factor to determine which improvement tracks they will be be placed on under the state’s new accountability system as outlined in its plan to comply with the federal Every Student Succeeds Act. For some schools, their TVAAS score will be the difference between continuing under a local intervention model or being eligible to enter the state-run Achievement School District. The school growth scores will also determine which charter schools are eligible for a new pot of state money for facilities.

The state has data analysts based across Tennessee to help districts with their questions and provide data simulations for the complex formula that has been replicated in other states.

“Of course, the reason it is complex is because we want it to be fair for educators and therefore capture as much data and nuance as possible – which is discussed at length in the technical documentation,” said a state department spokeswoman.

The state has also published an overview video of how the formula works and details on the recent changes in a 46-page, formula-packed document from SAS, the private company that calculates teacher and school scores for the state.

But as far as knowing how the state gets from A to Z, White said he still has questions.

“I’ve had some questions about getting access to certain data myself,” said White, who routinely interprets data for the district. “We would like a lot more access to what goes into TVAAS.” (He later declined to elaborate.)

He’s not the only one. When the Tennessee Education Association unsuccessfully sued Knox County Schools over its use of TVAAS in awarding teacher bonuses, access to data on how the scores were calculated was central to the association’s argument that the district denied teachers due process, said Rick Colbert, TEA’s general counsel.

When Colbert attempted to subpoena technical documents on the calculations, SAS blocked it partially because the request would divulge “trade secrets.”

“When they’re called upon to defend it you get a lot of general statements but you can’t get a lot of information to see if you can back that up,” Colbert said. “There’s so much about TVAAS that can’t be explained.”

Board member Mike Kernell called it a double standard and asked White last week if the district could request a demonstration of the complicated formula.

“I think the state department of education ought to show its work if they’re asking children to show their work,” he said.

making the rounds

Tennessee’s new education chief ‘very confident’ that online testing will be smooth in April

PHOTO: Shelby County Schools
Tennessee's new education commissioner Penny Schwinn (second from left) met with Douglass High School students and Shelby County Schools leaders Friday.

As Tennessee’s new education commissioner wrapped up her second week on the job by visiting Shelby County, Penny Schwinn said she feels “very confident” the state has learned from its mistakes in online testing.

During the more than three-hour ride to Memphis on Friday, Schwinn said she continued to pour over documents showing evidence that the corrections the state department staff have put in place will work.

“I feel very confident that our team has looked into that,” she told reporters in a press conference after meeting with students. “They’re working with the vendor to ensure that testing is as smooth as possible this year.” Currently the state is working with Questar, who administered TNReady online last year.

She also said the state’s request for proposals from testing vendors, which is already months behind, will be released in about two weeks.

PHOTO: Shelby County Schools
From left: John Bush, principal of Douglass High School; Penny Schwinn, Tennessee Education Commissioner; and Joris Ray, interim superintendent for Shelby County Schools.

“No later than that,” she said. “We hope and expect to have a vendor in place before the end of the fiscal year,” in late June.

The day Schwinn was hired, she said getting state testing right would be her first priority. Three years of major technical failures have severely damaged the trust educators and parents have in the state’s test, TNReady. It is the main measure of how schools and teachers are doing, but state lawmakers exempted districts from most testing consequences in 2018.


From Schwinn’s first day on the job: Tennessee’s new education chief wants to ‘listen and learn’ with school visits


Just prior to talking with reporters, Schwinn said she heard “hard-hitting questions” from several students at Douglass High School in Memphis about what the state can do to improve education. Reporters were not allowed to attend the student discussion with Schwinn and some Shelby County Schools leaders.

Douglass High entered Shelby County Schools’ turnaround program, known as the iZone, in 2016 and saw high academic growth in its first year. But test scores fell this past year as the state wrestled with online malfunctions.

Timmy Becton Jr., a senior at Douglass High, said he hopes for fewer tests and more projects to demonstrate what a student has learned. Those kind of assessments, he said, can help a student connect what they are learning to their daily life.

PHOTO: Shelby County Schools
Tennessee’s new education commissioner met with students at Douglass High School and Shelby County Schools leaders.

“We figured it would be a different way to measure and see how much knowledge a student really has on a specific subject,” he told Chalkbeat after meeting with Schwinn during a student roundtable session. “It’s a good alternative to taking tests.”

He said he was “surprised and happy” to see Schwinn actively seek student perspectives.

“I really think that’s the most important part because students are the ones going to school every day,” Becton said. “So, if you want find a good perspective on how to solve a problem, it’s really great to talk to the people who are actively involved in it and the people who are actually experiencing these problems directly.”

The state’s annual testing window runs from April 15 to May 3.

School discipline

Michigan schools have expelled fewer students, but that may not be cause for celebration

PHOTO: Getty Images

Michigan schools have expelled far fewer students since the state enacted laws aimed at cutting back on expulsions. But an advocate who’s pushed for an end to zero-tolerance policies pointed out persistent problems and told elected state education leaders this week that, “We shouldn’t start celebrating yet.”

This is why: Peri Stone-Palmquist, executive director of the Ypsilanti-based Student Advocacy Center, told State Board of Education members that in the 18 months since the new laws took effect in 2017, expulsions have dropped 12 percent. But she’s concerned that too many school leaders don’t understand the law or are ignoring its requirements. And she believes some schools are finding other ways of kicking kids out of school without expelling them.

Michigan did away with zero-tolerance policies that had earned it a reputation for having some of the toughest disciplinary rules in the nation. In their place, lawmakers instituted new rules, such as requiring schools to consider seven factors — including a student’s age, disciplinary record, disability and seriousness of the incident — in making expulsion decisions.

“We have had districts and charters tell advocates that they would not consider the seven factors at all,” Stone-Palmquist said. Others aren’t sharing with parents and students how those seven factors were used. And she said there’s a general “lack of understanding of lesser interventions and the persistent belief that lengthy removals remain necessary.”

That’s a problem, she and others say, because of the negative consequences of kicking students out of school. Studies have shown that students kicked out of school are often missing out on an education and are more likely to get into trouble. Advocates also worry that expulsion exacerbates what they describe as a “school-to-prison” pipeline.

She said advocates are noticing that more students are receiving long suspensions, an indication that some schools are suspending students rather than expelling them. Hiding students in suspension data won’t work much longer, though. Michigan now requires schools to collect such data, which soon will be public.

Stone-Palmquist also said that some schools aren’t even going through the expulsion process, but simply referring students with discipline issues to “understaffed virtual settings.”

“Once again, the students who need the most get the least, and no one has to report it as an expulsion.”

Stone-Palmquist gave an example of a ninth-grader involved in a verbal altercation who was expelled for a long time for persistent disobedience, “despite our team lining up extensive community resources for him and despite the district never trying positive interventions with him.”

In another case, a fifth-grader was expelled for 180 days for spitting at another student who had done the same to them first. Stone-Palmquist said the seven factors weren’t considered.

“We were told at the appeal hearing that the student’s behaviors were too dangerous to consider lesser interventions.”

She and Kristin Totten, an education lawyer for the ACLU of Michigan, provided board members with statistics that some members found alarming. Totten noted that an ACLU review of data collected by the federal government shows that for every 100 students in Michigan, 38 days are lost due to suspension. In Oakland County, 26 days are lost for every 100 students. In Macomb County, it’s 35 days and in Wayne County, it’s 55 days.

One child who’s experienced trauma for years was repeatedly suspended from multiple schools. The 11-year-old has been diagnosed with post-traumatic stress disorder and attention deficit hyperactivity disorder. This school year, she’s been suspended for 94 days.

“Never once were the seven factors mentioned to her mother,” Totten said.

Stone-Palmquist asked board members to consider recommendations, including developing a model student code of conduct that incorporates the new rules, partnering with the advocacy center to request an attorney general’s opinion on what districts are required to do, and expanding data collection.

Tom McMillin, a member of the state board, asked whether the state should consider financial penalties, such as withholding some state aid.

“I’m a fierce advocate for local control. But in areas where the incentives might not be there to do what’s right … I’m fine with the state stepping in,” McMillin said.

Board member Pamela Pugh said she appreciated the push for the board to “move with great speed.” She said the data and stories provided are “compelling, as well as convincing.”

Stone-Palmquist said that despite her concerns, there have been some successes.

“Districts that used to automatically expel 180 days for fights, for instance, have partnered with us to dramatically reduce those removals with great outcomes,” she said. “We know alternatives are possible and that they actually help get to the root of the problem, prevent future wrongdoing and repair the harm.”

The Detroit school district didn’t come up during the hearing. But on the same day Stone-Palmquist presented to the state board, Detroit Superintendent Nikolai Vitti gave a presentation to his local board of education about what’s happened in the months since the district embarked on an effort to improve school culture by revising the student code of conduct, hiring deans for each school, and providing training on alternative discipline methods.

The bottom line: Vitti said that schools are booting out dramatically fewer students and greatly increasing alternative methods of discipline. In-school suspensions are up, given the push against out-of-school suspensions.

But the changes have also raised concerns. Some school staff have said the new rules are tying their hands. Vitti said it will take time for the changes to take hold, and he outlined some areas that need to improve, including more training.