research shows

Gates Foundation study paints bleak picture of teaching quality

The study measured teachers against the criteria in Charlotte Danielson's Framework for Effective Teaching rubric, which is used in New York as a tool for observing teachers. Teachers scored better at classroom management than they did on measures of higher-order instructional challenges, such as asking productive questions.

A historic look inside the nation’s classrooms, including some in New York City, painted a bleak picture, according to a report released by the Bill and Melinda Gates Foundation today.

The second installment of the foundation’s ambitious Measures of Effective Teaching study, the report focuses on the picture of teaching yielded by five different classroom observation tools. It also scrutinizes those tools themselves, concluding that they are valuable as a way to help teachers improve but only useful as evaluation tools when combined with measures of student learning known as value-added scores.

The conclusion is a strong endorsement of the Obama administration’s approach to improving teaching by implementing new evaluations of teachers that draw on both observations and value-added measures. New York State took this approach to overhauling its evaluation system when it applied for federal Race to the Top funding.

Among the group of five observation tools the foundation studied is the rubric now being piloted in New York City classrooms as part of stalled efforts to implement the changes to teacher evaluation, Charlotte Danielson’s Framework for Effective Teaching.

Through all five lenses, instruction looked mediocre in an overwhelming majority of more than 1,000 classrooms studied, the report concludes. There were some bright spots. Many teachers were scored relatively well for the aspect of teaching known as “classroom management” — keeping students well-behaved, making sure they are engaged.

But teachers often fell short when it came to other elements of teaching, such as facilitating discussions, speaking precisely about concepts, and carefully modeling skills that students need to master. These higher-order skill sets, the report notes, are crucial in order for students to meet the raised standards outlined in the Common Core.

The study is the most expansive known examination of instruction in the U.S., reviewing more than 1,000 teachers for this report and nearly 3,000 for the study. Its lead authors are the economists Thomas Kane, of Harvard, and Douglas Staiger of Dartmouth, although more than a dozens researchers contributed to the study.

The evaluations were conducted by trained evaluators, who watched clips from videotape of more than 1,000 teachers around the country and then judged whether the teaching exhibited certain traits outlined in the observation tools.

One complicated aspect of the study is that it doesn’t just ask what the observation tools have to say about teaching; it also asks whether those observation tools are good ways to measure teaching at all. The question is crucial to the contentious teacher quality debate.

Motivated by the Obama administration’s focus on improving teaching by improving the way teachers are evaluated, the teacher quality debate has been dominated by a search for a better evaluation tool. The idea is that if school districts could have a better way to sort teachers, then they could increase quality by rewarding those who are most effective and improving or removing those who are less effective.

The study offers a qualified endorsement of the five observation tools it studied, saying that they should be one of multiple evaluation measures but that no one observation tool should be a sole measure. While the study found that all five observation tools had a positive association with student achievement as measured by value-added scores, the associations were not perfect.

And the tools’ reliability was relatively low — lower, in some cases, than the famously volatile judgments of value-added measures. When different observers used the same tool to evaluate the same teacher, they sometimes gave very different scores.

But the report does endorse using the observation tools in combination with value-added measures, as New York’s new evaluation system does. When researchers combined multiple observation tools’ judgments of teachers together — and then combined those with the teachers’ value-added scores, the result was a view of a teacher that was more able to predict future student achievement, the report says.

A final complication worth noting is that the study’s ultimate arbiter of what makes a good evaluation tool is itself under heavy scrutiny. That arbiter is a teacher’s value-added score, an estimate that attempts to extrapolate the amount of student learning for which a teacher can be held responsible, excluding other factors such as a student’s family income level.

A study that was the subject of a story in today’s New York Times found that value-added scores indeed are useful predictors not only of student achievement, but other measures of life success. Researchers have cast doubts on value-added measures’ validity, citing a host of concerns from the measures’ volatility to whether a high value-added score reflects true student learning or simply effective test prep.

Though an overhaul of teacher evaluation in New York has been stalled by the failure of teachers unions and school districts to agree on how to conduct it, both the New York City teachers union and the Department of Education agreed to participate in the Gates Foundation study when it launched in 2009. The union helped recruit teachers to join, and ultimately, teachers from about 100 schools signed up to have their lessons videotaped and analyzed.

“It takes the politics out of what’s being measured,” UFT president Michael Mulgrew said when the union first agreed to participate. “Teachers are very frustrated with the political debate. They are always saying, ‘why don’t you just come into the classroom?’ That’s what this is doing.”

Since then, the politics over teacher quality has grown even more heated.

Last summer, a GothamSchools reader who had worked in a school piloting the Danielson evaluation said it was very hard for teachers to be rated “effective.”

Newcomers

This mom came to the newcomer school ready to stand up for son. She left with a job.

PHOTO: Provided by Javier Barrera Cervantes
Charlotte Uwimbabazi is a bilingual assistant at the Indianapolis Public Schools newcomer program.

When Charlotte Uwimbabazi showed up at Indianapolis Public Schools’ program for new immigrants, she was ready to fight for her son. When she left that day, she had a job.

A native of the Congo, Uwimbabazi fled the war-torn country and spent nearly a decade in Cape Town, South Africa waiting for a new home. Last spring, Uwimbabazi and her four children came to the U.S. as refugees.

(Read: Teaching when students are full of fear: Inside Indiana’s first school for new immigrants)

When the family arrived in Indianapolis, Uwimbabazi’s youngest son Dave enrolled at Northwest High School. But he wasn’t happy, and a volunteer helping to resettle the family suggested he transfer to Crispus Attucks High School. When the school year started, though, Crispus Attucks turned him away. Staff there said he had been assigned to the newcomer program, a school for students who are new to the country and still learning English.

There was just one problem: Dave was fluent in English after growing up in South Africa. Uwimbabazi said it’s the only language he knows. So Uwimbabazi and her son Dave headed to the newcomer school to convince them he should attend Crispus Attucks.

That’s when Jessica Feeser, who oversees the newcomer school and programing for IPS students learning English, stepped in — and found a new resource for the district’s growing population of newcomer students.

Feeser immediately realized that Dave was fluent in English and should enroll in Crispus Attucks. Then, Uwimbabazi started talking with Feeser about her own experience. With a gift for languages, Uwimbabazi speaks six fluently, including Swahili and Kinyarwanda, languages spoken by many African students at the school.

“Where are you working right now?” Feeser asked. “Would you like a job here?”

Uwimbabazi, who had been packing mail, took a job as a bilingual assistant at the school. Her interaction with the district went from “negativity to positivity,” she said.

The newcomer program has seen dramatic growth in enrollment since it opened last fall, and it serves about three dozen refugee students. Students at the school speak at least 14 different languages. As the only staff member at the school who speaks Swahili and Kinyarwanda, Uwimbabazi is a lifeline for many of the African refugees it serves, Feeser said.

She works alongside teachers, going over material in languages that students speak fluently and helping them grasp everything from simple instructions to complex concepts like graphing linear equations, Feeser said. She also helps bridge the divide between the district and the Congolese community on Indianapolis’ westside, going on home visits to meet parents and helping convince families to enroll their children in school.

When students in the newcomer program don’t share a language with staff members, the school is still able to educate them, Feeser said. But it is hard to build community without that bridge. Uwimbabazi has played an essential role in helping the school build relationships with families.

“She believes that all of our families are important, and she’s working diligently to make sure that they feel that their voices are heard,” Feeser said. “It was a gift from God that she joined us.”

the end

A 60-year-old group that places volunteers in New York City schools is shutting down

PHOTO: August Young

Citing a lack of support from the city education department, a 60-year-old nonprofit that places volunteers in New York City schools is closing its doors next month.

Learning Leaders will cease operations on March 15, its executive director, Jane Heaphy, announced in a letter to volunteers and parents last week.

In the message, she said the group had slashed its budget by more than a third, started charging “partnership fees” to participating schools, and explored merging with another nonprofit. But the city pitched in with less and less every year, with no guarantee of consistency, she said.

“This funding volatility has created insurmountable challenges to the long-term viability of our organization,” Heaphy wrote. “We regret the vacuum that will be created by our closure.”

The group — which began as part of the city school system but became its own nonprofit in the 1970s — says its volunteers work with more than 100,000 students in more than 300 schools every year, many of them faithfully. When then-84-year-old Carolyn Breidenbach became the group’s 2013 volunteer of the year, she had been helping at P.S. 198 on the Upper East Side daily for 12 years.

Heaphy’s full message to volunteers is below:

Dear [volunteer],

It is with a heavy heart that I write to inform you Learning Leaders will cease operations on March 15 of this year. This organization has worked diligently over the last few years to sustain our work of engaging families as Learning Leaders, but the funding landscape has become too challenging to keep our programs going. While we have been able to increase our revenues from a generous community of funders, we have ultimately come to the conclusion that without a consistent and significant base of funding from the NYC Department of Education, we cannot leverage foundation grants, individual donors, or school fees sufficiently to cover program costs.

In the face of growing financial challenges, Learning Leaders reduced its costs as thoughtfully as possible — and in ways that did not affect our program quality. Rather, we sought to deepen and continually improve our service to schools and families while eliminating all but the most necessary costs. These efforts reduced our budget by more than 35 percent.

At the same time, we sought greater public support for our work with schools and families across the city. We are grateful to the foundations and individual donors that have believed in our work and provided financial support to keep it going. We were gratified when schools stepped up to support our efforts through partnership fees. While these fees only covered a portion of our costs, the willingness of principals to find these funds within their extremely tight school budgets was a testament to the value of our work.

Throughout an extended period of financial restructuring Learning Leaders advocated strongly with the Mayor’s Office and the DOE [Department of Education] for a return to historical levels of NYC DOE support for parent volunteer training and capacity building workshops. While we received some NYC DOE funding this year, it was less than what we needed and was not part of an ongoing budget initiative that would allow us to count on regular funding in the coming years. Several efforts to negotiate a merger with another nonprofit stalled due to the lack of firm financial commitment from the DOE. Over time, this funding volatility has created insurmountable challenges to the long-term viability of our organization.

We regret the vacuum that will be created by our closure. If you have questions or concerns about opportunities and support for family engagement and parent volunteer training, you can contact the NYC DOE’s Division of Family and Community Engagement at (212) 374-4118 or FACE@schools.nyc.gov.

On behalf of the board of directors and all of us at Learning Leaders, I offer heartfelt thanks for your partnership. We are deeply grateful for your work to support public school students’ success. It is only with your dedication and commitment that we accomplished all that we did over the last 60 years. We take some solace in knowing that we’ve helped improve the chances of success for more than 100,000 students every year. The Learning Leaders board and staff have been honored to serve you and your school communities.
Sincerely,

Jane Heaphy
Executive Director