research shows

Gates Foundation study paints bleak picture of teaching quality

The study measured teachers against the criteria in Charlotte Danielson's Framework for Effective Teaching rubric, which is used in New York as a tool for observing teachers. Teachers scored better at classroom management than they did on measures of higher-order instructional challenges, such as asking productive questions.

A historic look inside the nation’s classrooms, including some in New York City, painted a bleak picture, according to a report released by the Bill and Melinda Gates Foundation today.

The second installment of the foundation’s ambitious Measures of Effective Teaching study, the report focuses on the picture of teaching yielded by five different classroom observation tools. It also scrutinizes those tools themselves, concluding that they are valuable as a way to help teachers improve but only useful as evaluation tools when combined with measures of student learning known as value-added scores.

The conclusion is a strong endorsement of the Obama administration’s approach to improving teaching by implementing new evaluations of teachers that draw on both observations and value-added measures. New York State took this approach to overhauling its evaluation system when it applied for federal Race to the Top funding.

Among the group of five observation tools the foundation studied is the rubric now being piloted in New York City classrooms as part of stalled efforts to implement the changes to teacher evaluation, Charlotte Danielson’s Framework for Effective Teaching.

Through all five lenses, instruction looked mediocre in an overwhelming majority of more than 1,000 classrooms studied, the report concludes. There were some bright spots. Many teachers were scored relatively well for the aspect of teaching known as “classroom management” — keeping students well-behaved, making sure they are engaged.

But teachers often fell short when it came to other elements of teaching, such as facilitating discussions, speaking precisely about concepts, and carefully modeling skills that students need to master. These higher-order skill sets, the report notes, are crucial in order for students to meet the raised standards outlined in the Common Core.

The study is the most expansive known examination of instruction in the U.S., reviewing more than 1,000 teachers for this report and nearly 3,000 for the study. Its lead authors are the economists Thomas Kane, of Harvard, and Douglas Staiger of Dartmouth, although more than a dozens researchers contributed to the study.

The evaluations were conducted by trained evaluators, who watched clips from videotape of more than 1,000 teachers around the country and then judged whether the teaching exhibited certain traits outlined in the observation tools.

One complicated aspect of the study is that it doesn’t just ask what the observation tools have to say about teaching; it also asks whether those observation tools are good ways to measure teaching at all. The question is crucial to the contentious teacher quality debate.

Motivated by the Obama administration’s focus on improving teaching by improving the way teachers are evaluated, the teacher quality debate has been dominated by a search for a better evaluation tool. The idea is that if school districts could have a better way to sort teachers, then they could increase quality by rewarding those who are most effective and improving or removing those who are less effective.

The study offers a qualified endorsement of the five observation tools it studied, saying that they should be one of multiple evaluation measures but that no one observation tool should be a sole measure. While the study found that all five observation tools had a positive association with student achievement as measured by value-added scores, the associations were not perfect.

And the tools’ reliability was relatively low — lower, in some cases, than the famously volatile judgments of value-added measures. When different observers used the same tool to evaluate the same teacher, they sometimes gave very different scores.

But the report does endorse using the observation tools in combination with value-added measures, as New York’s new evaluation system does. When researchers combined multiple observation tools’ judgments of teachers together — and then combined those with the teachers’ value-added scores, the result was a view of a teacher that was more able to predict future student achievement, the report says.

A final complication worth noting is that the study’s ultimate arbiter of what makes a good evaluation tool is itself under heavy scrutiny. That arbiter is a teacher’s value-added score, an estimate that attempts to extrapolate the amount of student learning for which a teacher can be held responsible, excluding other factors such as a student’s family income level.

A study that was the subject of a story in today’s New York Times found that value-added scores indeed are useful predictors not only of student achievement, but other measures of life success. Researchers have cast doubts on value-added measures’ validity, citing a host of concerns from the measures’ volatility to whether a high value-added score reflects true student learning or simply effective test prep.

Though an overhaul of teacher evaluation in New York has been stalled by the failure of teachers unions and school districts to agree on how to conduct it, both the New York City teachers union and the Department of Education agreed to participate in the Gates Foundation study when it launched in 2009. The union helped recruit teachers to join, and ultimately, teachers from about 100 schools signed up to have their lessons videotaped and analyzed.

“It takes the politics out of what’s being measured,” UFT president Michael Mulgrew said when the union first agreed to participate. “Teachers are very frustrated with the political debate. They are always saying, ‘why don’t you just come into the classroom?’ That’s what this is doing.”

Since then, the politics over teacher quality has grown even more heated.

Last summer, a GothamSchools reader who had worked in a school piloting the Danielson evaluation said it was very hard for teachers to be rated “effective.”

Team Memphis

How do you get teacher candidates to fall in love with Memphis? Shelby County Schools is taking them to a Grizzlies game.

PHOTO: Nikki Boertman/The Commercial Appeal
Memphis Grizzlies fans raise their growl towels during an NBA game at the FedEx Forum on April 25, 2013.

Home to one of the nation’s 25 largest school districts, Memphis has stepped up efforts in recent years to attract talented new teachers to a fast-changing education landscape, and now is including the city’s popular NBA basketball team as part of its playbook.

Shelby County Schools will kick off its hiring season this weekend by treating teacher candidates to dinner and a free game between the Memphis Grizzlies and Dallas Mavericks on Friday night at FedEx Forum.

A networking event will follow on Saturday at the Halloran Centre for Performing Arts & Education, a new downtown venue operated by the Orpheum Theatre to put arts and education center stage.

The activities are part of a first-ever “preview weekend” to fill openings for next school year in Tennessee’s largest district. Shelby County Schools typically hires between 600 to 800 teachers each year and is especially in need of special education and math teachers, said district spokeswoman Kristin Tallent.

Historically, the district has simply held recruitment fairs,” Tallent said Monday. “Through the weekend events, the district is hoping to expose potential teachers to our school district and some of the best that Memphis has to offer, which includes the Memphis Grizzlies.”

Teacher recruitment, development and retention has been a centerpiece of school reform efforts in Memphis since 2009 when the district won a seven-year, $90 million Gates Foundation grant that came to a close this school year. That grant, in partnership with the local nonprofit SchoolSeed, is helping to fund this weekend’s recruitment event. (To learn more about the influence of the Gates Foundation on Memphis public schools, read our special report).

The preview weekend comes as Superintendent Dorsey Hopson has proposed a budget without a shortfall or layoffs for the first time in years. The spending plan also includes $10.7 million for teacher raises to address inequities in the pay structure and shift to performance-based pay.

The district is asking teacher candidates to RSVP by Friday.

Parent-to-Para

How the Adams 14 school district is empowering parents to join the classroom

PHOTO: Yesenia Robles
A parent volunteer works with two kindergarteners on reading as part of a pilot program at Dupont Elementary School that is training parents to become paraprofessionals.

Raeann Javier would like to know what she can do to help her second-grader read better. Sometimes, sitting with her daughter, the best she could offer was, “You know how to do this.”

Javier, a single mother, also would love to land another job to earn more for her family.

A pilot program launched by Adams 14 School District in Commerce City may help her with both.

The school district is trying to build more knowledgeable, active parents through classes and volunteer time working with young students struggling to read. For those who are interested, the program also provides parents a path to become paraprofessionals, or teacher’s aides.

The initiative is one way the nearly 8,000-student suburban district — facing state intervention this year after years of poor academic performance — is trying to turn things around.

District surveys found parents were looking for ways to become more supportive.

Javier, one of 17 mothers in the program, said she already feels like she has become a more patient parent less than a month in. She also is interested in becoming a paraprofessional to supplement the income she earns as an at-home nurse.

“It’s a little bit tough. I make it work,” Javier said. “But this would really, really help.”

Other parents taking part in the pilot program already were volunteers at their kids’ schools.

“They usually just did the normal things like helping with copying or sorting papers,” said Jesse Martinez, Adams 14’s director for family and community engagement. “But we really wanted to change that dynamic. We wanted to pull in our parents to tap their potential and bring them in to support their children.”

One of the parent volunteers, Susana Torres, was an elementary school teacher for 10 years before coming to the United States. Now with three children in district schools, Torres jumped at the opportunity to get back into a classroom.

“This is my thing,” Torres said. “I love the program.”

Torres also helps other Spanish-speaking moms who are part of the program. She said that even though they don’t have the teaching background she does, the program has made it easy for all of them to learn to help kids. “All you need is a passion to make change,” she said.

Pat Almeida, the principal of Dupont Elementary, where the program is being piloted, said the goal is also to help more students become proficient in reading before third grade — especially those who are not far behind but just need a boost to get to grade level.

“We’re able to give them more repetition so they can apply that to their reading,” Almeida said. “If they’re able to have more repetition, their progress is going to be accelerated.”

Dupont Elementary is among the Adams 14 schools that is struggling, though the school isn’t yet facing sanctions like the district as a whole is this year.

District officials have been working on setting up reforms all year to present to the state as a suggestion for their corrective action, including getting help from an outside company for developing curriculum and testing. Increasing parental engagement through this and other new efforts, like having teachers visit families at home, are part of the work to improve the district.

The parent-to-para program is being funded with money from the Denver-based Rose Community Foundation (Rose also supports Chalkbeat) and Climb Higher Colorado, a coalition of advocacy groups that support strong academic standards and tests.

At Dupont, while the parent volunteers work with almost 75 students that they pull out of class for about an hour, teachers can spend the time in class working with students who need the most help.

An instructional coach supervises the moms to work with groups of two to six students and helps them plan lessons each day for kids.

During one lesson this week, parents were helping kindergarteners learn how to differentiate between capital and lowercase letters and how to sound out words. Some students were still having trouble identifying letters, while one boy was writing words so quickly he was standing up, moving around and at one point fell.

The volunteers said it’s rewarding to see the kids catching on.

“Knowing that just a little bit of our time can help them is a good feeling,” said volunteer Adelaida Guerrero. “It’s an excellent opportunity for them and for us.”

For Maria Rodriguez, the program has unexpectedly given her another benefit — bringing her closer to her teenage daughters. She said she joined the program because when a bilingual program for her two oldest daughters was removed seven years ago, she had stopped being able to help them on their school work.

When Rodriguez heard about the program, she thought she could prepare to help her younger children, a second and third grader, before they too required more help than she could offer.

“It’s brilliant,”Rodriguez said. “I’ve been helping them work on their vowels.”

Within the last week, the two older girls came to Rodriguez complaining that she hadn’t ever worked to help them in the same way, and asking to join in during the at-home lessons. Over time, the girls had kept their ability to speak Spanish, but never learned how to write it. Now they were asking to learn alongside their younger siblings.

“They have that apathy of adolescence that makes them not always want to get close to us as parents,” Rodriguez said, tearing up as she recalled the moment. “I honestly felt really good.”