Chalkbeat Ideas is a new section featuring reported columns on the big ideas and debates shaping American schools. Please join us for our next event: It’s about whether a four-year college degree is still worth it. With a panel featuring the CEO of the KIPP Foundation and a Princeton University economist, we’ll try to separate fact from fiction about the value of American higher education. This online event will be at 2 p.m. ET on March 16. You can sign up and submit questions here.
In 2009, the federal government launched a remarkable educational experiment. Effective teachers were paid large bonuses ($30,000, adjusted for inflation, over two years) to move into a low-performing, high-poverty school.
This reflected the moment’s zeitgeist: use test scores to identify the best teachers (and also the worst). This so-called Talent Transfer Initiative worked, according to a 2013 study: Test scores rose by 3 to 5 percentile points among students taught by transferring educators. It was an example of the tangible benefits of having an excellent teacher.
Yet the story does not end there. A group of researchers recently released a paper reexamining this study and offered an intriguing twist. If the transferring teachers remained as effective as they had been in their prior school, test scores would have risen even more.
But in fact, in their new schools, these great teachers transformed into merely pretty good teachers. This reflects a profound and sometimes underappreciated fact about teacher performance: It’s not just about the inherent skills of an individual. It’s also about the school environment.
“Teacher effectiveness is dynamic,” says Matthew Kraft, a Brown University professor and coauthor of the new paper. “Teaching is a team sport.”
The study is important because there’s been a vacuum in policy efforts to improve teacher quality since the 2010s. The bevy of teacher evaluation laws during that period produced little in the way of overall student learning gains, according to a widely cited study coauthored by Kraft. Teachers are still very important, though. This new study is one small step in understanding how to bolster the profession.
In the original study, the researchers recruited 10 large districts. Some low-performing schools within each district got the chance to hire highly effective teachers with the lure of large bonuses; other schools did not. Genuine experiments like this are rare in education.
The study relied on “value-added” scores, which are statistical estimates of how much an individual teacher contributes to classroomwide test score gains. These measures were highly controversial when used to evaluate teachers, but remain widely used by academics for research purposes. Teachers were eligible for the transfer bonus if they were in the top fifth of value-added scores within their grade and subject.
The study was done through the Institute of Education Sciences, the research arm of the Department of Education that has since been gutted by the Trump administration. Its future remains in flux.
Ultimately, 80 top teachers took the bonus and switched schools. In their new jobs, though, these teachers’ performance fell from the 85th percentile districtwide to the 66th percentile. (And for stats nerds, no, this probably wasn’t just regression to the mean.) So what happened?
Teachers’ context had dramatically changed. They reported lower student motivation and more behavior problems. Many more students were from low-income families, and their incoming test scores were lower. The value-added measure accounts for differences in student population, so it wasn’t inevitable that transferring teachers’ performance would fall. Teachers may simply have been less equipped and experienced for the challenges in their new schools.
Some of the challenges may have come from the schools themselves. Teachers reported having fewer resources and less autonomy over their classrooms compared to their prior school.
Other research has found that teachers perform better when they find a school they like, when they work with more effective peers, and when their school is conducive to learning and collaboration. Teachers also tend to do better when they’re teaching the same grade or the same group of students as in years past. Years of teaching experience matter a lot, too.
This adds nuance to the common sentiment that a child’s individual teacher is the most important in-school factor affecting learning. It’s really the teacher plus the various contexts that make it more or less likely that the teacher will succeed.
Teachers are not widgets in multiple senses. They cannot be simply moved around to different schools or classrooms or grades and expected to perform. “We need to move beyond thinking about teacher effectiveness as a fixed characteristic,” says Kraft.
Yet it’s not precisely clear how policymakers can take advantage of this insight. Yes, teachers do better in schools with strong leadership, good collaboration with colleagues, and support for handling student behavior challenges. Making sure all these things are in place is the tough part.
This new study hardly means that policymakers should give up on trying to use pay to get good teachers in high-poverty schools. The original Talent Transfer Initiative didn’t work as well as it might have, but it did make a difference. Kraft says it may have worked even better if the money was sustained over time and went toward retaining top teachers already working in those schools.
Matt Barnum is Chalkbeat’s ideas editor. Reach him at mbarnum@chalkbeat.org.
