light of day

City releases Teacher Data Reports — and a slew of caveats

When the Department of Education’s embargo of Teacher Data Reports details lifted at noon today, news organizations across the city rushed to make the data available.

The Teacher Data Reports are “value-added” assessments of teachers’ effectiveness that were produced from 2008 to 2010 for reading and math teachers in grades 3 to 8.

This morning, department officials including Chancellor Dennis Walcott and Chief Academic Officer Shael Polakow-Suransky met with reporters to offer caution about how the data reports should be used. They emphasized the reports’ wide margins of error — 35 percentage points for math teachers and 53 percentage points for reading teachers, on average — and that the reports reflect only a small portion of teachers’ work.

“We would never advise anyone — parent, reporter, principal, teacher — to draw a conclusion based on this score alone,” Polakow-Suransky said.

Most of the news organizations that filed Freedom of Information Law requests for the ratings plan to publish them in searchable or streamlined databases, with the teachers’ names attached. GothamSchools does not plan to publish the data with teachers’ names or identifying characteristics included because of concerns about the data’s reliability.

At least two other news organizations that cover education are also not publishing the data: the local affiliate of Fox News, according to a representative of Fox, and the nonprofit school information website Insideschools.

Department officials are asking schools not to release the reports to parents. They issued a guide today advising principals about how to handle parents who demand that their child be removed from the class of a teacher rated ineffective.

“Resist changing student/teacher assignments mid-year, as doing so is disruptive to all students’ learning,” the guide advises.

“We definitely will not be moving kids around based on a data point that is two years old,” Polakow-Suransky said today.

Joel Klein, who was chancellor when the reports were launched, was a public champion of the idea that releasing teacher ratings would empower parents to demand better schools and teachers for their children. He argued publicly that the ratings of individual teachers should be released after news organizations filed legal requests for them.

But Walcott began distancing himself from that position almost immediately after becoming chancellor last year. And last fall, his administration announced it had stopped producing the reports, citing a new state teacher evaluation system that will include value-added measures. Walcott explained in a Daily News column today that he is not thrilled that teachers “might be denigrated” as a result of the data dump.

Today, Walcott and Polakow-Suransky argued that Klein’s initial stance was justified — at the time. “What we’re looking at in 2012 is quite different from what Joel was looking at in 2010,” Polakow-Suransky said.

Because New York City adopted value-added measures relatively early, he said, the ratings were in fact among the most sophisticated measures available at the time. But even at the time, the ratings were not meant for public consumption or to be used in high-stakes decision-making, he said.

Polakow-Suransky acknowledged one exception to the high-stakes decision ban: School officials did use value-added ratings to inform decisions about whether to grant some teachers tenure.

We reported last year that principals said they were being prevented from granting tenure to teachers whose students had low test scores, particularly if they were among the roughly 12,000 teachers annually to receive a data report.

Polakow-Suransky said today that 133 teachers up for tenure last year were flagged because of low value-added scores. More than a third of the flagged teachers received tenure after their principals and superintendents determined that other factors carried more weight, he said.

The city released reports for about 18,000 different teachers over the course of the three school years during which they were produced. About 12,000 teachers received reports in each of the 2008-2009 and 2009-2010 school years, and about half of them received two because they taught both reading and math. (A smaller pilot group received reports in 2007-2008.)

Because many teachers’ students were tested in both math and English, some teachers were given two different value-added ratings for a single year — below average in one subject and above average in the other.

The reports also indicate whether teachers were found to have done especially well at boosting the test scores of particular types of students, such as students with disabilities, low-income students, or students who are considered English language learners.

Department officials said 77 percent of the teachers who received reports are still working in the city schools.

Next week, ratings will be released for teachers in schools for students with severe disabilities and for charter school teachers. The release of charter school teachers’ data reports reverses the department’s earlier decision not to release them because charter schools had opted into the Teacher Data Report program voluntarily.

Officials cautioned that the scores were especially unreliable for teachers with relatively few students. That group includes elementary school teachers, who typically had fewer than 30 students factor into their scores.

They also said reports are less reliable for teachers whose students were either all high-performing or mostly low-performing. The difference is due to the nature of the state tests, which were designed to distinguish among middle-level students — those just above and below the state’s proficiency cutoff. As a result, small differences among students at the top and bottom had an outsized impact on a teachers’ rating, Polakow-Suransky said.

The issue was one of several lessons learned that Polakow-Suransky said the city has flagged for the State Education Department, which is building its own value-added algorithm to be used as a part of new teacher evaluations.

The state is working with the American Institute of Research to build its model; the city’s model was designed by a research center at the University of Wisconsin.

Other lessons Polakow-Suransky identified include the importance of publishing value-added scores in the context of their margins of error and the need to create a way for teachers and principals to verify data.

The city introduced a verification system that teachers could use two years into the Teacher Data Report project. Less than 40 percent of teachers signed in to use it, but a significant number of them found major errors. For example, three percent of the teachers who signed in discovered that they had been marked as teaching classes that they had not actually taught.

In their lawsuit attempting to stop the ratings from being released, United Federation of Teachers officials identified more than 200 such errors. UFT President Michael Mulgrew said today that the errors showed that the city had not been respectful of teachers in developing or releasing the ratings.

“This was a complete debacle in terms of how the DOE has handled [the reports] and the mismanagement of the data inside of the system,” he said.

Asked whether he thought the state’s model would avoid the pitfalls the UFT identified in the city’s reports, Mulgrew said, “I’m not confident.”

But he said the state’s evaluation requirements are an improvement over the Teacher Data Reports because the new evaluations will showcase the value-added calculations alongside other measures of teacher quality.

the end

A 60-year-old group that places volunteers in New York City schools is shutting down

PHOTO: August Young

Citing a lack of support from the city education department, a 60-year-old nonprofit that places volunteers in New York City schools is closing its doors next month.

Learning Leaders will cease operations on March 15, its executive director, Jane Heaphy, announced in a letter to volunteers and parents last week.

In the message, she said the group had slashed its budget by more than a third, started charging “partnership fees” to participating schools, and explored merging with another nonprofit. But the city pitched in with less and less every year, with no guarantee of consistency, she said.

“This funding volatility has created insurmountable challenges to the long-term viability of our organization,” Heaphy wrote. “We regret the vacuum that will be created by our closure.”

The group — which began as part of the city school system but became its own nonprofit in the 1970s — says its volunteers work with more than 100,000 students in more than 300 schools every year, many of them faithfully. When then-84-year-old Carolyn Breidenbach became the group’s 2013 volunteer of the year, she had been helping at P.S. 198 on the Upper East Side daily for 12 years.

Heaphy’s full message to volunteers is below:

Dear [volunteer],

It is with a heavy heart that I write to inform you Learning Leaders will cease operations on March 15 of this year. This organization has worked diligently over the last few years to sustain our work of engaging families as Learning Leaders, but the funding landscape has become too challenging to keep our programs going. While we have been able to increase our revenues from a generous community of funders, we have ultimately come to the conclusion that without a consistent and significant base of funding from the NYC Department of Education, we cannot leverage foundation grants, individual donors, or school fees sufficiently to cover program costs.

In the face of growing financial challenges, Learning Leaders reduced its costs as thoughtfully as possible — and in ways that did not affect our program quality. Rather, we sought to deepen and continually improve our service to schools and families while eliminating all but the most necessary costs. These efforts reduced our budget by more than 35 percent.

At the same time, we sought greater public support for our work with schools and families across the city. We are grateful to the foundations and individual donors that have believed in our work and provided financial support to keep it going. We were gratified when schools stepped up to support our efforts through partnership fees. While these fees only covered a portion of our costs, the willingness of principals to find these funds within their extremely tight school budgets was a testament to the value of our work.

Throughout an extended period of financial restructuring Learning Leaders advocated strongly with the Mayor’s Office and the DOE [Department of Education] for a return to historical levels of NYC DOE support for parent volunteer training and capacity building workshops. While we received some NYC DOE funding this year, it was less than what we needed and was not part of an ongoing budget initiative that would allow us to count on regular funding in the coming years. Several efforts to negotiate a merger with another nonprofit stalled due to the lack of firm financial commitment from the DOE. Over time, this funding volatility has created insurmountable challenges to the long-term viability of our organization.

We regret the vacuum that will be created by our closure. If you have questions or concerns about opportunities and support for family engagement and parent volunteer training, you can contact the NYC DOE’s Division of Family and Community Engagement at (212) 374-4118 or [email protected].

On behalf of the board of directors and all of us at Learning Leaders, I offer heartfelt thanks for your partnership. We are deeply grateful for your work to support public school students’ success. It is only with your dedication and commitment that we accomplished all that we did over the last 60 years. We take some solace in knowing that we’ve helped improve the chances of success for more than 100,000 students every year. The Learning Leaders board and staff have been honored to serve you and your school communities.
Sincerely,

Jane Heaphy
Executive Director

Rise & Shine

While you were waking up, the U.S. Senate took a big step toward confirming Betsy DeVos as education secretary

Betsy DeVos’s confirmation as education secretary is all but assured after an unusual and contentious early-morning vote by the U.S. Senate.

The Senate convened at 6:30 a.m. Friday to “invoke cloture” on DeVos’s embattled nomination, a move meant to end a debate that has grown unusually pitched both within the lawmaking body and in the wider public.

They voted 52-48 to advance her nomination, teeing up a final confirmation vote by the end of the day Monday.

Two Republican senators who said earlier this week that they would not vote to confirm DeVos joined their colleagues in voting to allow a final vote on Monday. Susan Collins of Maine and Lisa Murkowski of Alaska cited DeVos’s lack of experience in public education and the knowledge gaps she displayed during her confirmation hearing last month when announcing their decisions and each said feedback from constituents had informed their decisions.

Americans across the country have been flooding their senators with phone calls, faxes, and in-person visits to share opposition to DeVos, a Michigan philanthropist who has been a leading advocate for school vouchers but who has never worked in public education.

They are likely to keep up the pressure over the weekend and through the final vote, which could be decided by a tie-breaking vote by Vice President Mike Pence.

Two senators commented on the debate after the vote. Republican Lamar Alexander of Tennessee, who has been a leading cheerleader for DeVos, said he “couldn’t understand” criticism of programs that let families choose their schools.

But Democrat Patty Murray of Washington repeated the many critiques of DeVos that she has heard from constituents. She also said she was “extremely disappointed” in the confirmation process, including the early-morning debate-ending vote.

“Right from the start it was very clear that Republicans intended to jam this nomination through … Corners were cut, precedents were ignored, debate was cut off, and reasonable requests and questions were blocked,” she said. “I’ve never seen anything like it.”