evaluating evaluation

More work, worse relationships, and better feedback: How teacher evaluation has changed the job of the principal

PHOTO: Laura Faith Kebede/Chalkbeat
Principal Melody Smith discusses how students at A.B. Hill Elementary grew significantly in test scores.

Teacher evaluation overhauls were supposed to reshape the teaching profession. New research suggests they may have had an even greater impact on what it means to be a school principal.

As policy makers overhauled teacher rating systems in the last decade, principals began spending much more of their time watching teachers in action and talking to them about how to improve. But the shift also overwhelmed them with work, stopped them from fulfilling other responsibilities in their schools, and weakened their relationships with teachers.

Those are some of the takeaways of a study based on interviews with dozens of principals in six districts that revamped how teachers were evaluated in the last decade. Typically, the new evaluation systems — often put in place at the urging of the federal government and influential philanthropies, particularly the Gates Foundation — incorporated student performance for the first time but were driven mostly by teachers’ scores on rubrics that attempt to spell out good teaching practices.

While the systems’ adoption often precipitated significant pushback and in some cases have been rolled back, they remain in force in many places — and their long-term consequences are only now becoming clear.

The study, published last week in The Elementary School Journal, focuses on six large districts that embarked on dramatic evaluation changes: Baltimore, Denver, Hillsborough County (Florida), Houston, Memphis, and Metro Nashville. The researchers interviewed 10 principals in each district for about an hour during the 2012-13 school years, selecting a group at random while also ensuring a combination in different grade and performance levels.

One finding stood out. “The role of administrator is drastically changing,” one principal said. “You cannot be just a manager of a building. … You have to be an instructional leader first.”

The study examines only a handful of the many districts that reworked evaluations in the last decade, so it’s unclear how widely the findings apply. But the researchers detected themes across the six locations, which all had somewhat different evaluation systems and strategies for rolling them out. Here’s what they heard.

The new evaluations take way more time than principals feel they have.

The most consistent concern shared with researchers centered on the time it takes to complete the new evaluations. The responsibilities didn’t just include observing teachers more frequently, but also conducting pre- and post-observation conferences, completing additional paperwork and data entry, and using final ratings to make human resources decisions.

“I mean, honest to God. I just can’t do it by myself,” one principal told the researchers. “If I’m running my school, something’s going to lose out, either academics or your school because you’re just one person.”

“We need to [have] a preconference, an observation, and a post conference, you’re talking about, even though a preconference and postconference may only take 20 to 30 minutes and then an observation, you’re looking at two hours,” another principal said.

“It took us like an hour and a half to input the evaluations per teacher, and I have 82 people. So it was like a nightmare, literally,” said another.

In New York City, where the recently proposed teachers union contract reduces the number of required observation from four to two for most teachers, principals say the blanket requirements of the new system impeded their ability to make good choices about how they spend their time.

“We’re better off spending more time with the teachers who need our help than with the teachers who are doing a bang-up job,” the president of the union that represents school administrators told Chalkbeat. “To say time is at a premium for our members is an understatement.”

Principals are spending more time with teachers — and many say that’s a good thing.

Although the use of test scores has driven much of recent public debates about evaluations, the principals interviewed in the study often focused on the role of observations. That’s in line with past research suggesting principals trust that measure more than test scores; it’s also not surprising since principals conducted observations themselves, and those scores accounted for the lion’s share of a teacher’s overall rating.

While some frameworks for evaluating instruction have gotten mixed reviews, principals in the study said they found them helpful. Having an “objective” rubric, the principals told the researchers, allowed them to both give specific feedback to teachers who were improving and to build evidence to dismiss teachers who persistently struggled.

“For me, as a leader, it has given me a tool that I can use to help teachers grow, more than I’ve ever been able to do in the past, ever, because it’s a lot more specific than it ever has been in the past,” one principal said.

“The … rubric is the single most solid evaluative tool that I’ve used in my career,” said another.

“The conversations and the reflection … have gone from, ‘Oh, I thought it went great,’ to a really robust conversation.”  said a third principal.

Whether observations are really as “objective” as principals seemed to believe is not clear. Other research has shown that classroom observations are consistently biased against teachers who serve more students of color and students who start the year with lower test scores.

Principals’ relationships with teachers worsened.

Though there has been modest uptick in teachers who received low ratings, the vast majority of teachers across the country were still rated effective under the new systems.

Still, principals said the new evaluations damaged their relationships with teachers, who weren’t always sold that the new approaches were fair. They cited this dynamic as a major downside of new evaluation systems.

Some principals framed the tension as a lack of understanding on teachers’ part. “It’s very difficult for some teachers to understand that you really can quantify what they’re doing in the classroom, and many of them don’t want you quantifying it,” one said.

Others said the culture in the school had changed as teachers became more fearful of high-stakes evaluation, and thus less open with their principal.

“I feel like every time I walk in, a teacher’s like automatically on pins and needles thinking I’m there in an evaluative capacity,” a principal told researchers. “I just want to go in and see what the kids are doing.”

Principals ended up being less visible in their schools.

One surprising finding is that even as the new evaluation systems required principals to spend lots of time in classrooms, they said they ended up being less visible to educators and students.

“I’m in more classrooms, but I’m not walking through lots of classes. I’m mainly only in four classes or three classes a day,” one principal explained.

“I don’t do lunch duty as much,” another said. “I loved lunch duty because it’s a time to interact with the kids where you can just communicate with them.”

Trisha Arnold, a New York City teacher who helped negotiate the contract terms that reduce required observations there, told Chalkbeat earlier this month that she had experienced this phenomenon in her school.

“Kids aren’t sure who the principal is,” she said, “because they’re bogged down with paperwork.”

What to do with the research is unclear.

How do the benefits and costs of new systems weigh out? That remains a frequently debated question among policymakers and researchers. A recent analysis focusing on the Gates Foundation’s evaluation reforms in a number of districts yielded disappointing results, with little if any gains in achievement. (Two of those districts were the same ones featured in the latest study, which was also funded by the Gates Foundation; additionally, Gates is a supporter of Chalkbeat.) Another study found that evaluations systems likely deterred prospective teachers from entering the profession.

Other research is more upbeat, linking evaluation to test score gains and teacher improvement.  Studies of evaluation efforts in Cincinnati, Chicago, Washington D.C., and Minnesota have all found benefits, as well as national research on performance pay and feedback connected to classroom observations.

The latest research suggests that finding ways to lighten principals’ workloads might be worthwhile. That’s already happened to a degree, with some principals saying they placed more responsibility on assistant principals or other administrators. A separate Washington State study found a hiring spree of assistant principals soon after evaluation systems were put into place. That might be a smart move to help principals — but it also costs money, which could have been spent elsewhere, say, increasing teacher pay or reducing class size.

The new research also underscores how significant the choice of observation rubric is for districts and schools that adopt new evaluation systems. “Policy makers may also want to consider the enormous weight that is being placed on instructional rubrics as a means of guiding principals’ understanding of teacher performance,” the authors say.

Perhaps most importantly, the authors say, there should be more attention on principals’ role in enacting the raft of procedures connected to new evaluations.

“Without a careful examination of how to address principals’ concerns about lack of time,” they conclude, “the risk of principal fatigue, frustration, and eventual turnover remains substantial.”

voices of the vote

Meet Denver teachers who voted yes to a strike, no to a strike — and just aren’t sure

PHOTO: PHOTO: Andy Cross/The Denver Post
Skinner Middle School math teacher Peter English walks out of the Riverside Baptist Church with his son, Landon, left, and daughter Brooke strapped to his chest after voting on whether to go on strike ()

Throughout the day, the parking lot of Riverside Baptist Church filled up as Denver teachers made their way into a meeting organized by their union, the Denver Classroom Teachers Association.  

Months of negotiations that failed to produce a deal between top leaders of Denver Public Schools and the union had given way to individual teachers facing a choice: To strike or not?

Along with reporting the news of the day — which you can read about here — Chalkbeat spent time visiting with teachers to get a sense of what was shaping their decision-making.

Most teachers we spoke with, both in depth and in passing, said they voted “yes” to strike. Union officials have said two-thirds of those who vote Saturday and in a second session Tuesday must sign off on a strike for it to proceed, and the prevailing wisdom among teachers we interviewed was that support is strong.

The decision, though, is far from black and white for many teachers, regardless of where they ultimately land.

Here are the stories of three teachers, all at different places:

Krista Skuce, Slavens K-8 school: Yes to strike

At the urging of teachers and parents, Slavens K-8 students turned out early on a few recent mornings to show support for their teachers. They wore red in solidarity and posed for pictures.

They also brought questions. “Why are you doing this?” was one.

Krista Skuce, a physical education teacher and 14-year Denver Public Schools employee, would tell students that she lives 40 minutes from the school because she can’t afford to live in Denver.

Krista Skuce

But there is more to her story. Her spouse, she said, is no longer able to work, beset by medical issues, unable to draw disability benefits, and in need of costly care including massage therapy, chiropractic appointments, neuromuscular therapies, and more.  

At the same time, Skuce said her pay “doesn’t seem to be going anywhere.” So she hustles, earning extra pay by driving across town to coach softball and basketball.

Skuce, like many teachers who stopped to talk about their votes on Saturday, believes the district can do more to boost teachers’ base pay — before bonuses and incentives.  

She said her salary has only increased $4,000 or $5,000 in the past 14 years, even though she has been teaching 27 years, has a master’s degree, and is National Board Certified.

Skuce said she knows that by voting to strike, she could very well end up taking money out of her own bank account. Striking teachers don’t get paychecks.

“I am hoping the district and the DCTA do the right thing and recognize the fact that there are some people here who are on the edge,” she said. “We are on the edge emotionally, financially. We know these are good people. And I think teachers are people who wake up every morning with forgiveness.

“You have to take a stand and say what you are for at some point in time in your life — and this is it,” she said. “I’m willing to do it, scary or not.”  

Jason Clymer, John F. Kennedy High School: No to strike

An English teacher at John F. Kennedy High School, Jason Clymer stands with his fellow union members in the belief teachers aren’t paid enough. He finds fault with what is asked of teachers through LEAP, the district’s growth and performance system for teachers.

“Teachers at my school feel extremely micromanaged and can’t catch a breath,” he said.  

But in the end, after being one of the first teachers in the door Saturday and attending an information session, Clymer said he voted against the strike.

“Going on strike is very hard,” said Clymer, whose wife works in human resources for the district’s central office. “And I think the agreement DPS came to was close enough.”

Clymer questioned picking a fight now because of the limited scope of the negotiations. That would be the current agreement governing ProComp, the pay system that provides teachers one-time bonuses for things like teaching in a high-poverty school, getting strong evaluations, having students who earn high test scores, or teaching in a high-performing school.

He said he’d like to save some political leverage to focus on other issues covered by the district’s main contract with the union.

“It’s really unfortunate these things can’t all be negotiated together,” he said. “If the district came out and said, ‘We want to give you more money, not as much as you like, but we want to devote more to things like mental health services,’ I really think that would be a winning argument.”

In opposing a strike, Clymer said that he did not want to divide his fellow teachers

“Although I voted no, I believe in the union,” he said. “And if the union voted to strike, I will absolutely support the union.”

Paula Zendle, Denver Green School: Undecided about strike

Paula Zendle is dreading the moment that is appearing increasingly likely: standing before her students at the Denver Green School and explaining why she won’t be there to teach them.

“I tell them constantly, ‘Don’t miss school, don’t miss school. Don’t be absent, don’t be absent, don’t be absent,’” said Zendle, her eyes welling up with tears as she waited on a friend. “I have been fighting to avoid a strike. I hate this. It’s utterly and totally agonizing to me.”

Paula Zendle

Zendle said she left a career in the corporate world for the classroom and has been teaching eight years. She teaches English language acquisition and Spanish at the Green School, a popular and highly-rated middle school option in a district that celebrates choice.

 Zendle said she has done her research and written to the district’s chief financial officer. What bothers her is a system she believes rewards younger teachers and underpays teachers in terms of the cost of living.  

The average Denver teacher currently earns about $51,000 in base pay and $57,000 with incentives, according to data from the state education department and the district. That’s less than teachers in districts like Boulder Valley, Cherry Creek, and Littleton.

District officials have agreed to put $20 million more into teacher compensation and defended their most recent offer on Saturday as “compelling.”

For Zendle, the prospect of facing her students — and that she works in a supportive school environment — is contributing to her struggle in deciding whether to vote “yes” to strike.

So if the moment does come, what will she tell her students?

“We have the right to protest unfair taxpayer spending,” she said. “This is not only unfair, it’s unconscionable. Their priorities have been wrong for 10 years.”

Then she paused and made clear that her decision had not been made. She considers herself a person of principle, and that will guide her in making a decision.

lesson plan

Denver hopes to keep its schools open in a strike — and the union wants you to send your kids

PHOTO: Kathryn Scott Osler/The Denver Post
Students eat lunch in the cafeteria at Dora Moore K-8 School in Denver.

Superintendent Susana Cordova says she is committed to keeping Denver schools open and continuing to educate students in the event of a strike.

In Los Angeles, where a teacher strike is entering its second week, many students are watching movies and playing games. Cordova said she plans to do more for the 71,000 students in district-run schools if teachers vote to strike and state intervention does not lead to a deal. The 21,000 students who attend charter schools will not be affected.

“We want to assure parents school will stay open,” she said. “We know it is critically important that we focus on the education of our kids. Sixty percent of our kids qualify for free and reduced-price lunch. We know they depend on school not just for their meals but for their access to opportunity.”

Negotiations broke down Friday between the district and the Denver Classroom Teachers Association, the union that represents teachers and special service providers such as nurses, school psychologists, and counselors. A strike vote is taking place in two sessions, one Saturday and another Tuesday. The earliest a strike could start is Jan. 28.

This would be the first strike in 25 years in the state’s largest school district. In 1994, the district used more than 1,000 substitutes to keep schools open, though many parents kept their children at home, something union leaders encouraged.

It’s not clear yet how high teacher participation in a strike would be. During the final week of bargaining, some teachers reported near universal support in their buildings, while others said some of their colleagues were uncertain. Some teachers have said they disagree with the union position in the negotiations and won’t participate as a matter of principle.

Teachers who strike do not get paid while they are not at work.

Cordova said the district is “in the process of building out our sub pool” and offering higher pay to those willing to work during a strike. But she declined to say how many substitutes the district could call on, and some teachers say they already have a hard time finding subs for routine absences.

Substitutes who work during a strike will earn $200 a day, double the normal rate, and “super subs” who work more than a certain number of days a year will get $250.

Many central office staff who have past teaching experience will be sent to schools to work with students. Cordova said the district is working on pre-packaged lesson plans for every grade and subject area so that learning can still take place, and officials will prioritize placing qualified staff members with special education and preschool students, those she deemed most vulnerable.

Students who get free or reduced-price lunch will still be able to eat in school cafeterias.

For its part, the union is encouraging parents to send their children to school, but with a different purpose.

“One major goal of a strike is for school buildings to be shut down as a demonstration of the essential labor performed by educators,” the union wrote in an FAQ document. “To this end, we encourage parents to send their students to school if their school building remains open. Student safety is paramount for all district schools, therefore the district will be obliged to close schools if safety becomes an issue due to limited staffing.”

Union officials said they were working to establish alternative daytime care with community partners like churches and Boys and Girls Clubs should schools close.