research shows

Gates Foundation study paints bleak picture of teaching quality

The study measured teachers against the criteria in Charlotte Danielson's Framework for Effective Teaching rubric, which is used in New York as a tool for observing teachers. Teachers scored better at classroom management than they did on measures of higher-order instructional challenges, such as asking productive questions.

A historic look inside the nation’s classrooms, including some in New York City, painted a bleak picture, according to a report released by the Bill and Melinda Gates Foundation today.

The second installment of the foundation’s ambitious Measures of Effective Teaching study, the report focuses on the picture of teaching yielded by five different classroom observation tools. It also scrutinizes those tools themselves, concluding that they are valuable as a way to help teachers improve but only useful as evaluation tools when combined with measures of student learning known as value-added scores.

The conclusion is a strong endorsement of the Obama administration’s approach to improving teaching by implementing new evaluations of teachers that draw on both observations and value-added measures. New York State took this approach to overhauling its evaluation system when it applied for federal Race to the Top funding.

Among the group of five observation tools the foundation studied is the rubric now being piloted in New York City classrooms as part of stalled efforts to implement the changes to teacher evaluation, Charlotte Danielson’s Framework for Effective Teaching.

Through all five lenses, instruction looked mediocre in an overwhelming majority of more than 1,000 classrooms studied, the report concludes. There were some bright spots. Many teachers were scored relatively well for the aspect of teaching known as “classroom management” — keeping students well-behaved, making sure they are engaged.

But teachers often fell short when it came to other elements of teaching, such as facilitating discussions, speaking precisely about concepts, and carefully modeling skills that students need to master. These higher-order skill sets, the report notes, are crucial in order for students to meet the raised standards outlined in the Common Core.

The study is the most expansive known examination of instruction in the U.S., reviewing more than 1,000 teachers for this report and nearly 3,000 for the study. Its lead authors are the economists Thomas Kane, of Harvard, and Douglas Staiger of Dartmouth, although more than a dozens researchers contributed to the study.

The evaluations were conducted by trained evaluators, who watched clips from videotape of more than 1,000 teachers around the country and then judged whether the teaching exhibited certain traits outlined in the observation tools.

One complicated aspect of the study is that it doesn’t just ask what the observation tools have to say about teaching; it also asks whether those observation tools are good ways to measure teaching at all. The question is crucial to the contentious teacher quality debate.

Motivated by the Obama administration’s focus on improving teaching by improving the way teachers are evaluated, the teacher quality debate has been dominated by a search for a better evaluation tool. The idea is that if school districts could have a better way to sort teachers, then they could increase quality by rewarding those who are most effective and improving or removing those who are less effective.

The study offers a qualified endorsement of the five observation tools it studied, saying that they should be one of multiple evaluation measures but that no one observation tool should be a sole measure. While the study found that all five observation tools had a positive association with student achievement as measured by value-added scores, the associations were not perfect.

And the tools’ reliability was relatively low — lower, in some cases, than the famously volatile judgments of value-added measures. When different observers used the same tool to evaluate the same teacher, they sometimes gave very different scores.

But the report does endorse using the observation tools in combination with value-added measures, as New York’s new evaluation system does. When researchers combined multiple observation tools’ judgments of teachers together — and then combined those with the teachers’ value-added scores, the result was a view of a teacher that was more able to predict future student achievement, the report says.

A final complication worth noting is that the study’s ultimate arbiter of what makes a good evaluation tool is itself under heavy scrutiny. That arbiter is a teacher’s value-added score, an estimate that attempts to extrapolate the amount of student learning for which a teacher can be held responsible, excluding other factors such as a student’s family income level.

A study that was the subject of a story in today’s New York Times found that value-added scores indeed are useful predictors not only of student achievement, but other measures of life success. Researchers have cast doubts on value-added measures’ validity, citing a host of concerns from the measures’ volatility to whether a high value-added score reflects true student learning or simply effective test prep.

Though an overhaul of teacher evaluation in New York has been stalled by the failure of teachers unions and school districts to agree on how to conduct it, both the New York City teachers union and the Department of Education agreed to participate in the Gates Foundation study when it launched in 2009. The union helped recruit teachers to join, and ultimately, teachers from about 100 schools signed up to have their lessons videotaped and analyzed.

“It takes the politics out of what’s being measured,” UFT president Michael Mulgrew said when the union first agreed to participate. “Teachers are very frustrated with the political debate. They are always saying, ‘why don’t you just come into the classroom?’ That’s what this is doing.”

Since then, the politics over teacher quality has grown even more heated.

Last summer, a GothamSchools reader who had worked in a school piloting the Danielson evaluation said it was very hard for teachers to be rated “effective.”

What's your education story?

Leaving finance was easy, teaching was hard, but this educator realized: ‘I’ve got to figure it out.’

PHOTO: Shaina Cavazos
Tom Hakin is an assistant principal at Cold Springs Elementary School in IPS.

education_story_graphic

Chalkbeat journalists ask the people we come across in our work to tell us about their education stories and how learning shaped who they are today. Learn more about this series, and read other installments, here.

Tom Hakim is now an assistant principal at Cold Springs Elementary School in Indianapolis Public Schools. He’s been an educator in Marion County for eight years.

I worked in corporate finance for four years — I was a finance undergrad — in Grand Rapids, Michigan. I was always involved and volunteered a lot in high school and college and did a lot of things in the community … (As an adult), I was a big brother in Big Brothers, Big Sisters, and I was coaching rec league basketball, I was teaching Junior Achievement, so that was actually my first experience going into a classroom teaching kids.

But through all of that, (those activities were) what I was really getting excited about in my days, not so much the day-to-day work I was doing in my current job.

So I just started looking at education and what options might be out there. And I read an article in the Detroit Free Press about Teach For America, and I had never heard about it prior to that. This was a chance for me to transition very quickly into a career that I think I may want to do, and that happened.

Even in a program like that, where you get the “meat and potatoes” of training of how to be a teacher, your first couple years, it’s just so hard.

But there was more of an idealistic commitment to it. This is what I think I really want to do, so despite the challenges, I’ve got to figure it out.

I was part of the 2009 (TFA) corp, taught in the charter world for five years here in town, and then had an opportunity three years ago to move to Washington Township. It’s actually the only place I’ve ever lived in Indy, and it was kind of the right fit at the right time.

I was a department chair at one of their middle schools. It was one that was the lowest performing at the time, Northview.

I think for the first time in my teaching career, I felt, in addition to working really hard and wanting to provide a great education for the kids of the school I was working in, I also felt that bigger commitment of the community because that was where I was living. My own children are going to school (there).

So it made it even more real for me, the “why” behind what we do.

I grew up outside of Detroit. When you really look at it, some of the issues that plague Detroit Public Schools are some of the same things we see here in Indy. My experience was not comparable to what I see some of our kids going through on a day-to-day basis.

So I think there’s the initial shock of that, but then there’s this next phase, where if I’m really going to be a part of this, I’ve got to understand all the factors that are really involved and going on here, and it’s not easy. It’s not easy work.

Over two years (at Northview), we went from lowest performing to highest math (test scores) in the district in my time as the department chair, which I primarily attribute to the team I had around me.

I was teaching one grade level, but again, it gets back to this idea that when you get the right teachers on board, growing in the same direction and pushing for the right things, pretty great things can happen.

unpaid leave debate

Colorado Supreme Court weighs challenge to law governing job protections for teachers

PHOTO: Denver Post file
The Colorado Supreme Court.

Are good veteran teachers still guaranteed jobs in Colorado, provided they don’t mess up?

The Colorado Supreme Court heard arguments Wednesday on that issue and others related to a landmark 2010 state law that changed the rules for teacher evaluations and assignments.

Lawyers for Denver Public Schools squared off against lawyers representing individual teachers in two separate lawsuits. One case was brought on behalf of seven current and former DPS teachers. It challenges a provision of the 2010 law that allows school districts to, under certain circumstances, put effective teachers who’ve earned job protections on unpaid leave.

The other case was filed by a single teacher, Lisa Johnson, who was put on unpaid leave.

In both cases, lawyers for DPS argued that putting experienced, effective teachers on unpaid leave is not the same as firing them — and thus doing so doesn’t violate their due process rights.

But lawyers for the teachers said unpaid leave is essentially “an end run” around those rights.

To understand both lawsuits, it helps to have some background on the 2010 law, known as Senate Bill 191. It did several things, including change the way teachers earn “non-probationary status,” which affords them job protections. To earn that status, teachers must now have three years of effective ratings instead of just three years of employment.

Earning that status is desirable because non-probationary teachers can only be fired for a limited number of reasons, including insubordination and unsatisfactory performance. In addition, non-probationary teachers are entitled to a hearing before being fired.

The 2010 law also effectively eliminated a practice known as “forced placement.” Before the law, teachers who lost their jobs not for cause but due to circumstances such as a decrease in student enrollment were assigned to open positions at other schools.

DPS officials didn’t like forced placement because most teachers were placed at low-income schools, which they said led to the neediest kids being taught by teachers who didn’t choose to be there. So after Senate Bill 191 passed, DPS changed its policy. The district now gives teachers who lose their positions temporary assignments with the expectation that they will look for “mutual consent” positions, meaning a school’s principal agrees to hire them.

If a teacher doesn’t find a mutual consent position within 18 months, he or she is placed on unpaid leave as per Senate Bill 191. The teacher is welcome to continue looking for jobs in DPS and is entitled to his or her previous salary and benefits if hired.

Since Senate Bill 191 went into effect, at least 1,113 non-probationary DPS teachers have lost their positions due to a decrease in student enrollment, the closure of a school or other similar circumstances listed in the law, according to data the district provided at Chalkbeat’s request.

The majority of them have found mutual consent positions. Sixty-two teachers are currently on unpaid leave because they were unable to do so, according to DPS.

However, that number doesn’t include teachers who resigned or retired rather than be put on unpaid leave. That information is difficult to gather, a district spokesman said, but DPS did tally some numbers in response to an open records request the Denver teachers union submitted in February. As best the district could tell as of earlier this year, 39 non-probationary teachers who lost their positions between 2010 and 2014 resigned and seven retired.

On Wednesday, a lawyer for the teachers who brought the lawsuit argued that state law has historically afforded teachers a “basic bargain:” if they work for three years and are asked to come back for a fourth, they’re entitled to job protections. Lawmakers were wrong to alter that under Senate Bill 191, attorney Philip Hostak told the seven justices.

But DPS’s lawyer pointed out that the historical idea of tenure no longer exists — and hasn’t since lawmakers stripped the word from state law in 1990. “There is no indication in the legislation itself … that these folks are permanent teachers,” said attorney Eric Hall.

Pushing back on Hostak’s argument, Hall said lawmakers can amend laws however they see fit — and in the case of Senate Bill 191, they added the mutual consent provision and unpaid leave.

A lawyer for Johnson also challenged that provision and argued that Johnson shouldn’t be subject to it because she lost her position for a reason not listed in Senate Bill 191.

“The legislature tells us exactly which teachers can be displaced,” said attorney Eric Harrington.

However, the lawyer representing DPS in the Johnson case, Jonathan Fero, argued that the reasons listed in the law aren’t exhaustive and mutual consent applies to all teachers.

The lawyers did not debate the reason Johnson lost her position except to say they disagreed on the facts but that those facts aren’t an issue for the Supreme Court to decide.

The justices typically take months to issue an opinion.