high stakes

There’s always been confusion surrounding Tennessee’s growth model. With a missing year of data, new questions pile on

At a time when scores are about to be used for high-stakes decisions in how to improve Tennessee’s schools, gaps in the state’s data and uncertainty about how scores were derived have left Memphis officials wondering how to interpret the torrent of information.

Last year’s chaotic state testing, which led to the cancellation of the state’s test for grades 3 to 8, left a crucial gap in the data meant to help make decisions about schools and teachers.

School leaders have also said they were puzzled by the state’s methodology in reaching the so-called growth scores upon which districts and schools are judged — particularly by how they arrived at the Memphis district’s low score.

Even those who are paid to sift through the data say they are having trouble getting answers to questions about the growth scores, known as TVAAS. Bill White, chief of planning and accountability for Shelby County Schools, conceded to board members last week that he didn’t know the ins and outs of the complex formula and the changes meant to compensate for the missing data.

“I have personally never been shown all the mathematics behind our data and how this works,” he told board members. “I do know that it has been peer-reviewed and vetted and it’s essentially been held up among those statisticians. But there is a lot that goes on behind the scenes that no one has been able to walk us through.”

The confusion has renewed skepticism about the state’s value-added model, which is supposed to help officials identify the impact that schools and teachers have on student performance. The system relies on the state’s data measuring student growth in districts.

Part of the problem is last year’s botched testing, which is having multiple ripple effects throughout the state.

This year, growth scores are comparing 2016-17 test results with the 2014-15 school year, the most recent data available. That throws a wrench in how to assess which school or teacher is responsible for a child’s growth over a two-year period. And for elementary schools, that means there is no data for fourth graders this year since testing in third grade, the first year students take state tests, was canceled.

In addition, one subject was dropped entirely from TVAAS calculations because social studies questions were a trial run for elementary and middle schools students and did not count.

Statisticians for the most part have figured out how to calculate growth even when a state transitions to a new test. But the missing data creates a whole other host of challenges the revisions attempt to account for.

One Memphis charter leader said he still isn’t quite sure how his school even got a score since last year his highest grade level at the school was third grade, the first year of testing.

“It’s such a convoluted formula, it’s hard for us to understand. We’re not sure how we got (our score),” said the charter leader, who declined to be named because he was still seeking answers from the state.

Damian Betebenner, a senior associate at Center for Assessment that regularly consults with state departments, said missing data on top of a testing transition “muddies the water” on results.

“When you look at growth over two years, so how much the student grew from third to fifth grade, then it’s probably going to be a meaningful quantity,” he said. “But to then assert that it isolates the school contribution becomes a pretty tenuous assertion… It adds another thing that’s changing underneath the scene.”

At the same time, TVAAS scores for struggling schools will be a significant factor to determine which improvement tracks they will be be placed on under the state’s new accountability system as outlined in its plan to comply with the federal Every Student Succeeds Act. For some schools, their TVAAS score will be the difference between continuing under a local intervention model or being eligible to enter the state-run Achievement School District. The school growth scores will also determine which charter schools are eligible for a new pot of state money for facilities.

The state has data analysts based across Tennessee to help districts with their questions and provide data simulations for the complex formula that has been replicated in other states.

“Of course, the reason it is complex is because we want it to be fair for educators and therefore capture as much data and nuance as possible – which is discussed at length in the technical documentation,” said a state department spokeswoman.

The state has also published an overview video of how the formula works and details on the recent changes in a 46-page, formula-packed document from SAS, the private company that calculates teacher and school scores for the state.

But as far as knowing how the state gets from A to Z, White said he still has questions.

“I’ve had some questions about getting access to certain data myself,” said White, who routinely interprets data for the district. “We would like a lot more access to what goes into TVAAS.” (He later declined to elaborate.)

He’s not the only one. When the Tennessee Education Association unsuccessfully sued Knox County Schools over its use of TVAAS in awarding teacher bonuses, access to data on how the scores were calculated was central to the association’s argument that the district denied teachers due process, said Rick Colbert, TEA’s general counsel.

When Colbert attempted to subpoena technical documents on the calculations, SAS blocked it partially because the request would divulge “trade secrets.”

“When they’re called upon to defend it you get a lot of general statements but you can’t get a lot of information to see if you can back that up,” Colbert said. “There’s so much about TVAAS that can’t be explained.”

Board member Mike Kernell called it a double standard and asked White last week if the district could request a demonstration of the complicated formula.

“I think the state department of education ought to show its work if they’re asking children to show their work,” he said.

Measure of Success

State ratings identify 163 Colorado schools in need of improvement

PHOTO: AAron Ontiveroz/The Denver Post
Students in kindergarten on the first day of school at McGlone Academy.

More than 160 Colorado schools received one of the state’s two lowest ratings, making them eligible for additional assistance but also vulnerable to intervention if they don’t improve student performance.

The watch list comprises 9 percent of Colorado’s 1,800 schools and educate roughly 74,000 students, or 8.5 percent or the state’s almost 900,000 students. That means the vast majority of students in the state attend a school with one of the two higher rankings on the four-point scale.

The State Board of Education finalized the ratings Wednesday. The state gives separate district-wide ratings, which were finalized last month.

“The state’s accountability system is built on the premise that all students should receive a high quality education and graduate ready for college or careers,” Katy Anthes, Colorado’s education commissioner, said in a statement. “Our goal is to give all students a chance to excel. These designations allow us to identify struggling schools that may need more support to help students achieve their highest aspirations. And they also highlight successful schools so that other schools can learn from them.”

All public schools receive a state rating, known as the School Performance Framework report, each year. It’s based largely on student scores on the state’s English and math tests. Student growth, or how much students learn year-to-year compared to peers with similar results on state tests, carries most weight. High school graduation and dropout rates are also factored in.

Colorado Department of Education

There are four ratings: performance (the highest), improvement, priority improvement and turnaround (the lowest).

Schools and districts that have one of the lower two ratings are placed on a watch list and have five years to improve before facing state intervention. Schools on the list are eligible for grants for leadership training and help from outside consultants, but if change doesn’t come fast enough, the state could hand over control to an external manager, require conversion to a charter, or close schools.

Earlier this fall, the State Board of Education ordered the Adams 14 school district, based in Commerce City, and two schools in Pueblo in southern Colorado to turn over control to external managers after earlier intervention efforts did not produce enough improvement.

Colorado is still figuring out what effective intervention looks like and if outsiders can make a difference for students that existing leadership has not been able to achieve.

Most Colorado schools maintained the same rating they had in 2017, with 15 percent moving down at least one level and 14 percent moving up at least one level. Eighteen schools improved enough to get off the state watch list, which is often known as the “accountability clock,” some after initial state intervention last year.

Six schools are entering their eighth year on the watch list: Aurora Central High School, Adams City High School, Aguilar Junior-Senior High School in the tiny Aguilar district in southern Colorado, Hope Online Learning Academy Elementary School in Douglas County, Heroes Middle School, and Risley International Academy of Innovation, the last two both in Pueblo.

Two are entering year six: Central Elementary School in the Adams 14 district and Minnequa Elementary School in Pueblo.

Another four are entering year five, now the last year to improve before state intervention: Manual High School and Montbello Career and Technical High School in Denver, Mesa Elementary in the Montezuma-Cortez district in southwest Colorado, and EDCSD: Colorado Cyber School in Douglas County.

In the past, some schools received more time to improve because the “clock” was paused for several years as the state changed assessments. But now there are no more extensions beyond year five.

Of the state’s 42 online schools, a little more than half received one of the top two ratings, and 31 percent did not report enough data for the state to grant a rating. Colorado has more stringent regulations of online schools than many states, but there is an ongoing debate about how well these schools serve students.

About 84 percent of the state’s 247 charter schools received one of the top two ratings, compared to 89 percent of all Colorado schools. Twenty-six charter schools, or 10.5 percent, received one of the lowest two ratings.

Look up your school here:

Literacy tutors needed

Detroit enlists volunteer tutors before third-grade reading law takes effect

PHOTO: Anthony Lanzilote

Detroit’s school district is asking the community for help getting students reading at grade level. The superintendent is hoping volunteer literacy tutors will prevent a critical mass of third-graders from being held back under the state’s tough new reading law.

“We need your help,” Superintendent Nikolai Vitti said, making an appeal for volunteers during a school board meeting Tuesday night. “Our teachers and our principals and our schools alone will not be able to ensure that every student is at third-grade level without your help.”

Which is why the district is working with two community advocacy groups, Keep the Vote/No Takeover and the National Action Network, to launch the Let’s Read program, geared to K-3 students. The program is slated to begin in February — less than a year before the reading law takes effect. Once it does, during the 2019-2020 school year, Michigan third-graders who aren’t reading at grade level will be held back.

In the Detroit district, where proficiency levels on state exams are extremely low, the consequences could be dire. During a community forum last week, Vitti said that the law could hold back as many as 90% of Detroit third-graders, though Michigan’s education department has yet to define what it means for a student to be reading at grade level. At the forum, though, he noted exemptions from the law for such as students with special education needs and those who speak little to no English.

The Let’s Read volunteers will be assigned to individual students based on need. They will read with the children and help them with book selections.

Helen Moore, a longtime community activist who represents the two community organizations behind the volunteer effort, urged people to sign up during the public comment period of the meeting.

“I know our students will succeed, because they’re brilliant,” Moore said. But they and their parents need help, she said.

Vitti said the volunteer cohort is one of many literacy-building efforts underway. In addition, he said that every district school will hold family literacy nights and that its Parent Academy will expand its classes that teach parents how to help their children with reading. A community-wide event to teach Detroiters about the reading law — and what they can do to help — will also be held.

Moore said the word is starting to get out about the Let’s Read program, noting: “The telephone has been ringing like crazy. And now the suburban districts want to be part of it.”

The focus, though, is on Detroit, she said.

Want to volunteer: You can fill out a form here, or call 313-873-7884.