team of rivals

Seven ed research heavyweights to head to Albany to help direct evaluation overhaul

Education research heavyweights are headed to Albany next week to offer their advice about the state’s imminent overhaul of teacher evaluations, and they represent many sides of the contentious debates around how to rate teachers.

Seven researchers, economists, and professors accepted an invitation to weigh in on the debate at a May 7 summit being held by the State Education Department and Board of Regents, according to a department memo obtained by Chalkbeat. Officials are required by law to collect public comment on how to design the regulations that will govern how a teacher’s performance in the classroom gets graded, a process that must be complete by the end of June.

On Gov. Andrew Cuomo’s side is Thomas Kane, an economist from Harvard University who last week penned an op-ed praising the governor and the evaluation system he pushed into law earlier this year. Kane is best known for directing the Bill and Melinda Gates Foundation’s Measures of Effective Teaching study, an influential project that many states drew from when designing their new evaluation systems over the last half-decade. (Disclosure: Chalkbeat also receives some funding from the Gates Foundation.)

Kane’s three-year, $45 million project ultimately recommended that 33 to 50 percent of a teacher’s rating be determined by student growth as determined by their state test scores, 25 percent by student surveys, and the rest by classroom observations. (New York’s new system puts a heavier emphasis on student growth, but prohibits the use of student surveys.)

Kane will have some intellectual allies on the panel, but he’ll also be joined by several experts who are less supportive of Cuomo’s ideas. They include University of California at Berkley economist Jesse Rothstein, who analyzed data from Kane’s MET study and found substantial differences in value-added scores for the same teacher when different tests were used. Rothstein has also questioned whether state tests are the best assessments to capture teacher quality.

“If it’s right that some teachers are good at raising the state test scores and other teachers are good at raising other test scores, then we have to decide which tests we care about,” Rothstein told Chalkbeat in 2012. “If we’re not sure that this is the test that captures what good teaching is, then we might be getting our estimates of teaching quality very wrong.”

Also due to offer their opinions are Catherine Brown, vice president of the left-leaning think tank Center for American Progress, which has published papers endorsing the use of value-added methodologies; Manhattanville College professor Stephen Caldas, who once called New York’s evaluation system “psychometrically indefensible“; and Teachers College sociologist Aaron Pallas, who has been critical of New York’s system and served as an expert witness in lawsuits brought by teachers unions challenging low teacher ratings.

Balancing out the panel will be Sandi Jacobs, a vice president at the National Council on Teacher Quality, an organization that has pushed states to adopt stringent evaluation systems that rely more on student learning measures, and Leslie Guggenheim of TNTP, an advocacy organization whose influential 2009 paper “The Widget Effect” was critical of districts for not using teacher performance to make important policy decisions.

It’s unclear what influence the researchers and policy analysts will have on the state’s work, given that much of the evaluation system has been prescribed by the law passed in April. Still, officials have decisions to make about what types of student performance should be factored into evaluation plans and how those scores should be used.

According to the memo, the state’s invitation was turned down by several other prominent researchers. One was Stanford’s Linda Darling-Hammond, who said in an email that she was working with the California’s education department and couldn’t travel. Another was Sean Corcoran of New York University, who has found that the city’s teacher ratings calculated by value-added measures were highly volatile from one year to the next and often riddled with errors, and said he was traveling.

Six of the seven experts will appear in person next week, where representatives from teacher unions, superintendent associations, and other advocacy groups will also be in attendance. A spokesperson for the department said the experts will get about 45 minutes to present their views on teacher evaluation policy before taking questions from Regents members.

Meanwhile, the city teachers union got a jump start on its attempts to influence state education officials, sending a lengthy letter and documents to the Regents this week outlining its own goals for changes to teacher evaluations.

The letter stated for the first time that the union wants the state to eliminate the use of “group measures” of student growth, which can be used to give ratings to teachers based on test scores of students or subjects they do not teach. Those measures are among the most frustrating aspects of evaluations for teachers of non-tested subjects like art, music and physical education, but also allow schools to reduce the time students spend taking subject-specific tests.

The union also pushed for as many aspects of the evaluation system as possible be left to districts and their unions to negotiate, rather than be decided by state officials.

“Every district is different, and a top-down, one-size-fits-all system will not meet the needs of all students and schools,” Evelyn De Jesus, the union’s vice president for education wrote.

Teaching teachers

Year-long residencies for teachers are the hot new thing in teacher prep. But do they work?

For years, education advocates, policymakers and scholars have been trying to put an end to the underprepared novice teacher. The hope has been to find a training model that is just right, pairing theoretical knowledge and practical skills necessary for the messy reality of the classroom.

Now some think they’ve found an approach that works: teacher residencies.

Writing in the New York Times, three staff members of Bank Street College argued for this idea, comparing it to how doctors are trained.

“Aspiring teachers need well-designed and well-supported preparation,” wrote Shael Polakow-Suransky, Josh Thomases, and Karen DeMoss. “Year-long co-teaching residencies, where candidates work alongside an accomplished teacher while studying child development and teaching methods, offer a promising path.”

Indeed, there is consistent research showing that teachers trained through residencies are more likely to stay in the profession, potentially reducing churn in schools and costs of finding and training new teachers.

“When it’s done well, it’s kind of a solution to the teacher shortage problem that has plagued urban districts,” said Linda Darling-Hammond, president of the Learning Policy Institute, an education think tank that recently put out a report praising the residency model.

But preliminary new research focusing on Denver’s residency program showed that teachers trained through the program were less effective at improving student achievement in math than other novice teachers in Denver.

This echoes the findings of a study on the Boston Teacher Residency, a prominent example of the approach. In that case, residency-trained teachers also were less effective in math in their first years in the classroom — though they improved fairly quickly.

Together, the positive impacts on teacher retention and the more tepid effectiveness results might still suggest that residencies are worthwhile. But some see the enthusiasm getting ahead of the evidence, particularly in light of the steep price tag of such models.

“I am amazed by how much enthusiasm this idea seems to be generating, despite the fact that we don’t have much evidence to support it,” said Marty West, a Harvard professor who studied the Boston program.

A teacher residency has several key components, according to proponents.

Darling-Hammond’s group identifies several characteristics: a full year of student teaching under an experienced, effective mentor; a partnership between a school district and university so that practice and theory are closely linked; continuing mentorship after candidates become full-fledged teachers; and payment of student teachers during the residency year in return for a three- to four-year teaching commitment.

The final aspect is part of what makes the program appealing as well as costly.

“As I think about the common elements of residency program, there’s a lot that seems very promising — if also, potentially, very expensive,” West said.

Under a traditional university training model, students pay tuition; under the residency model they get paid, albeit modestly. The Boston Teacher Residency, for instance, is free for those who teach in Boston for three years, and offers candidates a $12,600 stipend as well as health insurance for their residency year. (In that program, teachers do have to pay tuition to UMass Boston to receive a master’s degree as part of the program.)

Retention rates of teachers trained through the Denver Teacher Residency

The upside is that those who go through residencies seem to remain teachers in their school districts for longer. In Denver, for instance, residents were 16 percentage points more likely than other novice teachers to return to the district. A national study of 12 teacher residency programs also showed higher retention rates.

This, Darling-Hammond hypothesizes, is explained by the quality of residence programs.

“I think that amount of student teaching and the mentor teacher being a true expert probably has a lot to do with the retention rate being strong,” she said. “You’re getting everything a beginning teacher should get.”

Although research on what makes teacher training effective has generally not come to clear conclusions, there is evidence for the idea that giving teachers practice in a real classroom is important.

But when it comes to the initial effectiveness of residency-trained teachers — at least as measured by the impact on students’ standardized English and math test scores — the evidence is mixed, and in some cases even negative.

West and colleagues found that teachers who go through the Boston Teacher Residency program were initially less effective at improving student achievement in math and no better in English, compared to other beginning teachers.

To West, these findings were counterintuitive.

“I was excited about the opportunity to evaluate the Boston Teacher Residency because I was optimistic,” he said. “I was surprised by our finding that residents were less effective than other new hires, at least initially.”

Darling-Hammond points out — and West agrees — that the teaching corps is likely to be particularly strong in Boston, where there is a robust higher education sector, so that it might be especially difficult for one program to prove particularly effective.

The Denver study, though, produced similar results: negative impacts on former residents’ students in math and essentially no effect in reading.

But there were bright spots in both evaluations. The teachers in Boston improved swiftly over time to the point that those teaching for five years were more effective than other experienced teachers. Combined with the lower turnover rates, the study estimates that the program had a modestly beneficial effect on student achievement over the long run.

And in Denver, the researchers also examined teachers’ classroom ratings, assigned by trained observers. There, former residents came out ahead of other teachers.

Other research on residency programs is thin but paints a more positive picture. A report on the New Visions Hunter College teacher residency in New York City showed that their teachers outperformed other novices in five high school exam areas, but underperformed in three others. A recent state analysis of 40 teachers trained through the Memphis Teacher Residency found they had above-average impacts on student test scores.

West says he is still optimistic about the residency model. The key question, he says, is whether the costs of the program outweigh the benefits — but no such comprehensive analysis has been done.

Darling-Hammond notes that some programs have tried to save costs by, for instance, using residents as substitutes one day a week or having them take the place of teacher aides. She also emphasizes the impact, financial and otherwise, of residencies on reducing teacher attrition.

“If you think about the costs of replacing teachers … this ends up being a cost-effective investment,” she said.

Training time

Common Core is out. Tennessee Academic Standards are in. Here’s how teachers are prepping for the change.

PHOTO: Marta W. Aldrich
Karyn Bailey (left), a facilitator from Williamson County Schools, coaches elementary school teachers during an exercise on Tennessee's revised standards for English language arts as part of a two-day training at La Vergne High School, one of 11 training sites across the state.

Teachers poring over Tennessee’s revised academic standards are mostly breathing a sigh of relief as the state prepares for its third change in eight years of what students are expected to learn in each grade.

The Tennessee Academic Standards for math and English language arts, which will reach K-12 classrooms this fall, aren’t dramatically different from Common Core standards used in Tennessee since 2012. But numerous tweaks are sprinkled across the standards — mostly word adjustments for clarity and changes in presentation to make the standards more user-friendly. Some standards also have been moved to different grades and courses for the sake of progression and manageability.

About 6,000 teachers got a two-day crash course on the changes during trainings last week at 11 sites across the state. The Tennessee Department of Education organized the sessions, attended by educators from more than 90 percent of Tennessee’s 146 districts.

“This is the third time I’ve gone through this, and it’s the best one from my perspective,” said John Lasater, a Sumner County math teacher who attended sessions at La Vergne High School near Nashville.

Lasater, who teaches at Westmoreland High School, was thrilled that some standards have been moved out of standards-heavy algebra classes into higher-level math courses. “It was just too much, especially Algebra II,” he said. “Our teachers just never seemed to be able to cover everything.”

Diving into her manual for English language arts, Rutherford County teacher Leila Hinkle liked seeing a greater emphasis on early writing skills, as well as the embedding of language standards in foundational literacy standards.

“I think the new standards are clearer; they’re clarifying,” said Hinkle, who will teach fourth grade this fall. “You can see better where students were supposed to be and where they’re going.”

Standards are foundational because they set learning goals that dictate other education decisions around curriculum and testing. In Tennessee, they are usually reviewed every six years.

The trainings are part of the last major step in a transition that began in 2014 when Gov. Bill Haslam ordered a review after Common Core became embroiled in political controversy over charges of federal overreach, in part because of incentives the Obama administration offered to states that adopted them. Eighteen months of review and revisions followed, with the State Board of Education approving the newly minted Tennessee Academic Standards last year.

The changes aren’t as drastic as in 2011 when Tennessee switched to Common Core. That’s because the committee of educators charged with the overhaul used the Common Core as a foundation rather than starting from scratch. Both sets of standards emphasize critical thinking and analysis and de-emphasize memorization of facts.

Education Commissioner Candice McQueen said many of the changes, while seemingly subtle, are no less substantive.

“A word change can be significant in terms of standards. If you change the word know to explain as far as what students must be able to do, that’s significant. That’s two different levels of understanding,” she said.

The trainings and training resources are costing the state about $3 million over the course of a year, far less than the $23.5 million in federal Race to the Top funds spent on Common Core trainings across three years.

To keep costs down, the state is changing its approach away from dependence on department-led trainings.

Erin McGill, a facilitator for trainings organized by the Tennessee Department of Education, dives into the revised standards with high school math teachers. (Photo by Marta W. Aldrich)

“We’re encouraging districts to identify teams to train with us and then to re-deliver the trainings in their own districts,” said Robbie Mitchell, the department’s executive director of academic strategy and operations. “It’s more about empowering and equipping districts to make their own decisions about what’s best for their district.”

The state plans to use the same training model for the rollout of new science standards in 2018 and social studies standards the following year.

Lasater said he found this year’s trainings productive and worthwhile.

“Before, the trainings were ‘here’s a standard, now let’s work a problem.’ This time, we were challenged to take a standard and develop a question that would fully assess a student’s mastery of it. We weren’t just working a problem; we were creating a problem. That’s a huge shift.”

You can find Tennessee’s new standards for math and English language arts on the Education Department’s website.