met-aphysics

Timely advice from Gates Foundation as evaluation talks resume

The Gates Foundation's latest report from its teacher-effectiveness study concludes that many evaluation models can be useful as long as they include multiple measures.

Now that the city and teachers union are back at the negotiating table to work on teacher evaluations, the Gates Foundation has some tips.

The foundation today released the third and final report about the Measures of Effective Teaching project, an ambitious three-year study that included 3,000 teachers in seven districts, including New York City. The study concludes that teacher effectiveness can indeed be measured and identifies strategies for grading teachers.

Having multiple people observe the same teacher is more effective than having one person observe the teacher multiple times, the study found. Student surveys are stronger predictors of teachers’ ability to raise test scores than observations. And counting state test scores for a third to half of a teacher’s rating is better than weighting the scores less or more.

With the report, the foundation takes a bold stance on a policy issue that remains hotly contested, even as states and school districts across the country have adopted new evaluation systems. But foundation officials are confident because the latest report reflects a change in the study’s design that they say proves that teacher evaluation systems really do measure teachers.

Last year, the foundation released a report concluding that evaluation systems that combine classroom observations and other measures, including student surveys, reliably predicted students’ test scores gains for the teachers in the study.

But the researchers couldn’t conclude that they were measuring the teacher’s effect with their evaluation tools. Those tools could have been picking up other characteristics of students, such as the stability of their home life, instead. The possible interference of unmeasurable influences has been one of the many critiques of “value-added” models, which aim to rate teachers based on a comparison of students’ predicted scores to their actual ones.

So the researchers asked the districts to let them assign the students in the first year of the study randomly to teachers in their school the following year. They concluded that four different evaluation models all showed, to varying degrees, that a teacher’s value-added score with one set of students is likely to be similar with a different set of students, at least within the same school. The study could not test whether teachers were just as likely to be effective in different settings, or with very different student populations.

The researchers also concluded that teachers who elevated their students’ scores on notoriously easy state tests also improved the students’ scores on tougher tests. The tougher tests were aligned to the Common Core standards, on which New York’s state tests will be based starting this spring.

City and union officials are working to hammer out an evaluation deal before next week, a state deadline for districts to adopt a new system or lose funding.

So far, New York has gotten some things right, according to the MET study. The state requires districts to use multiple measures — at least test scores and observations — to rate teachers. Using a rubric to assess observations is also mandatory, as are multiple observations.

But not everything the city and state are doing is what the Gates Foundation would want. State law requires only that 20 to 25 percent of each teacher’s rating to be based on state test scores, less than the study recommends. (Districts can opt to weight test scores more heavily, but few have done so.) The UFT has vowed not to allow student surveys to influence city teachers’ ratings. And the city Department of Education has pushed to make it optional for administrators to speak with teachers about the classes they observed, union officials have said.

Tequilla Banks, the leading educator in Memphis, Tenn., said there’s no point in observing teachers if the teachers and the people who observed them don’t speak about the experience together afterwards.

“That’s the most critical piece, honestly,” Banks said on a conference call today organized by the Gates Foundation. She added, “The observation itself is okay. The feedback and the post-conference, that’s the piece that [teachers] want.”

Still, Chancellor Dennis Walcott said in a statement that the study bore out the city’s approach to new evaluations.

“This report outlines exactly what the city has sought for our teachers and students: a fair evaluation system that looks at many factors, like classroom observations and student achievement. The study shows that evaluation systems can help teachers grow and learn — which in turn helps our students succeed,” he said.

Coming as the city and union scheduled talks for the first time in weeks, Walcott’s statement omitted a second goal the city has often cited for evaluations, beyond helping teachers improve: enabling the city to remove weak teachers.

That was appropriate, because Gates Foundation officials said the MET study has led them to conclude that meaningful teacher evaluations can improve classroom instruction over the long term — something the city’s own research has also borne out.

“Teaching is really complex and great practice takes time [and] tailored feedback,” said Vicki Phillips, the foundation’s education director, on the conference call. “Districts are better served by trying to improve practice rather than trying to make too-fine distinctions among teachers.”

A forthcoming teacher effectiveness project for the foundation will identify strategies for producing value-added scores for the vast majority of teachers whose students do not take state tests, according to Steve Cantrell, the foundation’s chief research officer.

Those findings will not come in time for the city and union to make use of them: They have to agree on how to rate teachers in non-tested grades and subjects before the state will accept the city’s evaluation system.

The Gates Foundation’s complete report is below:

Betsy DeVos

‘Underperformer,’ ‘bully,’ and a ‘mermaid with legs’: NYMag story slams Betsy DeVos

PHOTO: New York Magazine
A drawing of DeVos commissioned by an 8-year-old starts the New York Magazine article.

A new article detailing Betsy DeVos’s first six months as U.S. education secretary concludes that she’s “a mermaid with legs: clumsy, conspicuous, and unable to move forward.”

That’s just one of several brutal critiques of DeVos’s leadership and effectiveness in the New York Magazine story, by Lisa Miller, who has previously covered efforts to overhaul high schools, New York City’s pre-kindergarten push, and the apocalypse. Here are some highlights:

  • Bipartisan befuddlement: The story summarizes the left’s well known opposition to DeVos’s school choice agenda. But her political allies also say she’s making unnecessary mistakes: “Most mystifying to those invested in her success is why DeVos hasn’t found herself some better help.”
  • A friend’s defense: DeVos is “muzzled” by the Trump administration, said her friend and frequent defender Kevin Chavous, a school choice activist.
  • The department reacts: “More often than not press statements are being written by career staff,” a spokesperson told Miller, rejecting claims that politics are trumping policy concerns.
  • D.C. colleagues speak: “When you talk to her, it’s a blank stare,” said Charles Doolittle, who quit the Department of Education in June. A current education department employee says: “It’s not clear that the secretary is making decisions or really capable of understanding the elements of a good decision.”
  • Kids critique: The magazine commissioned six portraits of DeVos drawn by grade-schoolers.
  • Special Olympics flip-flop: DeVos started out saying she was proud to partner with the athletics competition for people with disabilities — and quickly turned to defending a budget that cuts the program’s funding.
  • In conclusion: DeVos is an underperformer,” a “bully” and “ineffective,” Miller found based on her reporting.

We’ve reached out for reaction from DeVos’s team and will update when we hear back.

teachers with borders

Schools near state lines perform worse — and rules discouraging teachers from moving may be to blame

PHOTO: Alan Petersime

Want a leg up in school? Don’t attend one near a state border.

That’s the surprising finding of a new study published in the Economics of Education Review. The likely culprit: certification and pension rules that discourage teachers from moving between states, limiting the labor pool on each side of the border.

The peer-reviewed paper focuses on test scores at public middle schools near a state boundary. Eighth-graders attending those schools, the researchers find, perform consistently worse in math than students at non-boundary schools. (The results are negative in reading, too, but smaller and not always statistically significant.)

One reason the findings ought to catch the attention of policymakers across the country: the data comes from 33 states, including big ones like Florida, New York, and Texas.

“We estimate that roughly 670,000 students are enrolled in middle schools nationally that are [considered] ‘intensely affected’ by a state boundary in our study,” the researchers write.

Of course, schools and students are not randomly assigned to be near state boundaries, so the study can’t definitively conclude that boundaries are the cause of lower performance. But the researchers — Dongwoo Kim, Cory Koedel, Shawn Ni, and Michael Podgursky, all of the University of Missouri — control for a number of student characteristics that might affect performance.

And while the study can’t pinpoint why a boundary seems to hurt test scores, the researchers have a theory: “state-specific pension and licensing policies” that discourage teachers from moving between states, likely forcing border schools to draw from a more limited pool of potential teachers.

In some places, those pension rules mean a substantial loss of retirement wealth if teachers move states mid-career. Complicated licensure rules that in some cases require experienced teachers to take certification exams or obtain additional degrees can also make that kind of switch practically difficult. Other research has found that teachers rarely move across state lines, even if they live near a boundary.

Why might that harm performance of schools near state lines?

Say a school in New York City has two science teachers and no math teachers, while a school right across the river in New Jersey has two math teachers and no science teachers. If each school needs exactly one teacher per subject, the solution is easy in theory: the New York City school gets a math teacher and loses a science one, and vice versa for the New Jersey school. But if certification or pension rules prevent that from happening, both schools lose out — and student achievement might suffer.

States aren’t typically eager to change those policies, though, for several reasons.

For one, states that require prospective teachers to clear a high bar to become certified may worry that making it too easy for an out-of-state teacher to receive a license could reduce teacher quality. A study from North Carolina provides some evidence for this argument, showing that teachers trained elsewhere were less effective than teachers trained in-state, though the difference was very small.

Another argument is that limiting teachers’ ability to bring pension money along with them when they move helps states hold on to their educators — even if they are in turn harmed when they can’t recruit teachers from elsewhere.

The latest study suggests that the net impact of those restrictions are negative. Still, the effects on students are quite small, implying that changes to pension and certification policies are unlikely to lead to large improvements in student performance.

But, the study points out, policies that eliminate the harm from attending school near a state line could help hundreds of thousands of students.

“Although the boundary effects are small on a per-student basis, they are spread across a very large population,” the researchers write.