value added?

Eight years ago, the L.A. Times published teachers’ ratings. New research tells us what happened next.

In 2010, the Los Angeles Times did something that hadn’t been done before.

The newspaper published test-score data for thousands of the city’s public school teachers, assigning them a rating based on how they influenced students’ results.

It caused a firestorm. Critics, including many teachers, railed against the measures as misleading and poorly constructed, warning that the data would demoralize teachers. The L.A. Times itself defended the release as necessary transparency.

In an accompanying story, one local teacher suggested it might also help children by empowering parents “to demand a good teacher.”

New research suggests that’s what happened next — but only for certain families.

Publishing the scores meant already high-achieving students were assigned to the classrooms of higher-rated teachers the next year, the study found. That could be because affluent or well-connected parents were able to pull strings to get their kids assigned to those top teachers, or because those teachers pushed to teach the highest-scoring students.

In other words, the academically rich got even richer — an unintended consequence of what could be considered a journalistic experiment in school reform.

“You shine a light on people who are underperforming and the hope is they improve,” said Jonah Rockoff, a professor at Columbia University who has studied these “value-added” measures. “But when you increase transparency, you may actually exacerbate inequality.”

That analysis is one of a number of studies to examine the lasting effects of the L.A. Times’ decision to publish those ratings eight years ago. Together, the results offer a new way of understanding a significant moment in the national debate over how to improve education, when bad teachers were seen as a central problem and more rigorous evaluations as a key solution.

The latest study, by Peter Bergman and Matthew Hill and published last month in the peer-reviewed journal Economics of Education Review, found that the publication of the ratings caused a one-year spike in teacher turnover. That’s not entirely surprising, considering many teachers felt attacked by the public airing of their ratings.

“Guilty as charged,” wrote one teacher with a low rating. “I am proud to be ‘less effective’ than some of my peers because I chose to teach to the emotional and academic needs of my students. In the future it seems I am being asked to put my public image first.”

But a separate study, by Nolan Pope at the University of Maryland, finds the publication of the ratings may have had some positive effects on students, perhaps by encouraging schools to better support struggling teachers.

Pope’s research showed that Los Angeles teachers’ performance, as measured by their value-added scores, improved after their scores were published. The effects were biggest for the teachers whose initial scores were lowest, and there was no evidence that the improvement was due to “teaching to the test.”

“These results suggest the public release of teacher ratings could raise the performance of low-rated teachers,” Pope concluded.

The two studies offer divergent pictures of the consequences of L.A. Times’ move. Pope did not find that higher-scoring students moved into the classrooms of higher-scoring teachers, while Bergman and Hill didn’t find clear evidence that teachers improved.

Those varying results are not entirely surprising, since the researchers used different methods. Pope’s research compared the same teachers before and after their value-added scores were published. Bergman and Hill took advantage of the fact that the L.A. Times only published scores for teachers who taught 60 or more students between 2003 and 2009, creating a natural experiment. The researchers then compared teachers who had taught just more than 60 kids to those teachers who had taught just under 60.

Rockoff of Columbia said he found both studies credible.

A third study, published in 2016, looks at an entirely different question: Did housing prices in Los Angeles increase near schools with more highly rated teachers?

Not really, according to the paper. That’s somewhat surprising, because past research has shown that housing zoned for schools with higher overall test scores and ratings is more expensive.

The researchers suggest that that might be because families had a hard time understanding what the ratings represented, and that some may have tuned out because of the surrounding controversy.

The results come in a very different political climate than around the time of the public release of the scores, when conversations about teacher performance had reached a fever pitch.

In 2010, then-Secretary of Education Arne Duncan praised the publication of teacher ratings. He used federal carrots and sticks to encourage states to use student test scores as part of how teachers are judged, a policy most states adopted.

But since then, states like New York and Virginia have barred the public release of this performance data, while media organizations have increasingly shied away from publicizing them. The new teacher evaluation systems have run into political challenges, and in some cases not had the hoped-for effects on student performance. And the federal education law passed in 2016 specifically banned the secretary of education from pushing teacher evaluation rules.

churning not learning

New research shows just how much losing a teacher midyear hurts students

PHOTO: Cyrus McCrimmon/Denver Post
Brown International Academy teacher Kate Tynan-Ridgeway works with a student.

The consequences of teacher churn were apparent to Esperanza Vazquez, a mother of two from New York City.

I had an experience with my son where he had a new teacher every week in math,” she told Chalkbeat recently. “That doesn’t help students.”

Now new research backs up Vazquez’s experience, documenting for perhaps the first time the steep consequences for students after teachers leave a classroom in middle of the school year.

The finding comes in a trio of new studies focusing on North Carolina. Together, they suggest that ill effects of teacher turnover identified in previous research may be driven largely by midyear departures; that those consequences extend even to students in the same grade whose teachers stay on; and that midyear turnover may be more common than previously thought, especially in schools serving more students of color and those from low-income families.

“While it is possible for turnover to be beneficial for school systems, an extensive body of research points to the ways that teacher turnover disrupts … the continuity of a child’s learning experiences, particularly in underserved schools,” write researchers Gary Henry of Vanderbilt and Christopher Redding of the University of Florida in one of the papers.

Henry and Redding’s three studies — two of which were published earlier this year in peer-reviewed journals, with the other is set to be published in coming weeks — home in on the rarely studied phenomenon of midyear teacher turnover.

Using recent data from North Carolina, two of the papers focus on the prevalence of the phenomenon. Annually 4.6 percent of teachers in the state departed midyear; among teachers in their first three years the rate jumped to 6 percent. The number was higher in schools deemed “underserved,” meaning they had more students of color and students in poverty, as well as lower test scores and fewer resources. Turnover was lower when principals were rated as more effective by teachers. It was higher among teachers who were less effective, those eligible for retirement benefits, and high school and middle school teachers.

Roughly a quarter of all teacher turnover in the state occurred in the middle of the school year.

The third study uses data from 2008 to 2014 to examine the consequences of midyear teacher attrition on elementary and middle school students’ test scores. In both math and English, students saw drops in learning as a result, controlling for a number of other factors. The decline in math scores was nearly as large as the difference in performance between an average teacher and an excellent one — a difference that has motivated dramatic policy changes in many places.

Impacts were smaller in English and in middle school, but also consistently negative. Students in the same grade level, but not class, of teachers were also harmed, but again less so.

The negative results are consistent with research on the effects of hiring teachers after the school year starts, in some ways a mirror image of the phenomenon.

The paper suggests three things that might explain the results: disruption in classrooms where teachers leave, instability in a school where teachers are exiting midyear, and less effective teachers replacing those who depart. The study suggests the first two theories seem to be clearly at play, since it was relatively ineffective teachers who were particularly likely to leave.

“When multiple teachers exit a school during the year, it can become increasingly difficult for teachers to maintain a work environment with a high degree of collaboration,” the researchers say.

The study did reach some surprising results: Students of color, students in poverty and students with lower prior test scores, generally did not suffer more as a result of midyear turnover; if anything, they suffered less in English. It may be that their schools were better prepared for midyear exits since they happen more frequently; it could also be that those students were simply “not well served by the teacher who departed,” the paper hypothesizes.

Another counterintuitive result: Unlike midyear turnover, departure of teachers at the end of the school year did not lead to declines in student learning, and even led to small benefits in some cases. That’s surprising in light of past research — and conventional wisdom — suggesting that teacher turnover harms students. (Prior studies generally have not distinguished between midyear and end-of-year turnover.)

The latest research does come with a key caveat: Test scores might be lower in classes where teachers leave midyear for other reasons — perhaps a particularly disruptive class causes both a teacher to quit and students to learn less in school. The authors attempt to account for this by comparing how the same student did in years when their teacher does not turnover.

The studies also look at just a single state, so it’s unclear whether the results would look similar elsewhere.

The researchers point out that some churn is inevitable, even healthy. “Many of the personal factors driving within-year teacher turnover are unlikely to be amenable to change: a teacher takes time in the middle of the school year for parental leave; a veteran teacher retires midyear; a beginning teacher leaves a few months into the school year after realizing teaching is a poor occupational fit,” write Henry and Redding. Indeed, female teachers between the ages of 26 and 40 years old were particularly likely to exit mid-year, indicating that parental leave plays a significant role in the results.

But the studies collectively conclude that students could benefit from combating midyear departures — although the best way to do that is not clear.

In Detroit, some schools have adopted a crude — and some would say cruel — approach, imposing financial penalties for teachers who left midyear. Studies focusing on turnover in general have found that higher pay, better working conditions, and more effective principals can make a difference.

At the same, time Henry and Redding argue that policymakers ought to make extensive efforts to avoid midyear teacher turnover when possible. For instance, they point out that information from teacher evaluation systems, including “value-added” test scores measures, aren’t always available until after the school year has begun. Finalizing those results before classes are underway could decrease midyear exits, they speculate.

“All measures of teachers’ performance, including their value-added scores, should be provided during the summer to allow teachers and administrators to attend to employment decisions without disrupting classes that have already begun,” the researchers conclude.

First Person

Staying ‘neutral’ after the Jason Van Dyke verdict was a tough ask of Chicago teachers like me

PHOTO: Joshua Lott/Getty Images
A woman holds a sign outside the courthouse after a murder verdict is handed down in the trial of Chicago Police Officer Jason Van Dyke.

At the beginning of every school year, I introduce myself to my students with a very personal presentation.

I show them pictures of where I grew up, my family, and the students I’ve taught at two other Chicago schools. I’m a human, not a robot, I tell them, earning a couple of laughs with my corny robot impression. At the end, I show them a signed copy of John Lewis’s “March,” the graphic novel that illustrates his experiences as a Civil Rights Movement leader. I talk about seeing Lewis speak at a Chicago Public Schools event years ago, and how he inspired me to speak up when I saw injustice.

In return, I ask my students to introduce themselves. They bring pictures of their lives, families, friends, and travels, and they talk about who they want to become. These presentations help to turn the library and writing center I oversee into a community.

The connection I have with my students isn’t out of the ordinary in Chicago. I’d be hard-pressed to find a teacher in the three very different high schools I’ve taught in and in schools all across the city who didn’t have strong ties to the students they teach. That’s why it felt so problematic that my district, CPS, asked its teachers to remain “neutral” about the Van Dyke case — the trial of a Chicago police officer, Jason Van Dyke, who was convicted of second-degree murder for shooting a Chicago teenager.

Two days before the Van Dyke decision came down, amid warnings that riots could follow a “not guilty” decision, the district sent an email advising teachers about how to handle discussions surrounding the verdict. I applaud the email for its initial statement: “It is critical that educators are prepared and provide space for students should they and their students choose to engage in this critical and timely public issue,” it read.

But in the next paragraph, the district said that teachers “must remain neutral.” The email cited a 2007 Indiana circuit court decision, Mayer v. Monroe County Schools, that ruled that “teachers do not have the constitutional right to introduce their own political views to students, ‘but must stick to the prescribed curriculum.’”

That left me with several questions. First, is an opinion on the Van Dyke trial truly a political view? Many of my students, now juniors and seniors, were just becoming teenagers when they watched the dash-cam video where 16 bullets riddle Laquan McDonald’s 17-year-old body.  The opinion CPS is concerned about me sharing, presumably, is that Van Dyke should face consequences.

I have taught many students like Laquan McDonald, students who have grown up in foster homes, who have failed out of the very school I taught in, whose city literally left them behind. When I saw Laquan McDonald in that video, I saw their faces grimacing on the ground, their bodies writhing. To me, his death, the subsequent cover-up, and the verdict, is personal, not political.

Asking me to “stay neutral” as a white teacher in a classroom full of African-American and Latinx students is asking me to send a message that I am indifferent to their experiences and to have them see me as a stereotype of whiteness. I am on their side. I don’t think there is anything wrong with having them know this. But the message I received implies that my district does.

In the court case evidently used as proof for why staying neutral is mandatory, students asked their teacher whether she ever protested. She told them that she honked her car horn at demonstrators calling for peace at an anti-Iraq War demonstration. The teacher believed she was fired because of this discussion. The court ruled in favor of the school district, stating, “the First Amendment does not entitle primary and secondary teachers, when conducting the education of captive audiences, to cover topics, or advocate viewpoints, that depart from the curriculum adopted by the school system.”

How this applies in my situation is confusing, since every year in my career I have either had the freedom to construct or co-construct curriculum for my classroom. It also reminds me of what Holocaust survivor and award-winning author Elie Wiesel said in his speech when winning the Nobel Peace Prize: “Neutrality helps the oppressor, never the victim. Silence encourages the tormentor, never the tormented. Sometimes we must interfere.”

Asking teachers to remain neutral when discussing Laquan McDonald teaches my students something I don’t want them ever to learn: that my connections with them, and my pursuit of justice for our shared community, are not my highest priority.

Gina Caneva is a 15-year Chicago Public Schools veteran who works as a teacher-librarian and Writing Center Director at Lindblom Math and Science Academy.  She is a National Board Certified teacher and Teach Plus Teaching Policy Fellowship alum. Follow her on Twitter @GinaCaneva.