5 lessons from Chalkbeat's event on teacher evaluations by video

The panel discussion I moderated on Tuesday evening focused on two video clips of classroom teaching that together lasted just over four minutes. Those four minutes fueled an hourlong discussion about how teachers are getting observed, especially now that New York City is implementing a teacher evaluation system that requires principals to spend far more time inside classrooms than ever before.

Our first two guests, a teacher and assistant principal from a middle school in the South Bronx, discussed quirks of the Danielson Framework, the way a subtle classroom command can derail a lesson and when technology trumps person. A policy analyst with New America NYC, Chalkbeat’s co-host for the event, then joined to offer a perspective on efforts to improve teacher quality around the country. An active online chat for the duration of the event added extra insight.

A summary of the event is below. If you missed it, we taped saved a live broadcast, which you can see (if not always hear) in its entirety here.

1. Danielson sometimes sets too high a bar for teachers 

Even a perfect teacher would have a hard time matching up to the hypothetical classroom educators described in Danielson’s rubric. In Danielson’s ideal, students of “highly effective” teachers finish work early and ask for more, offer one another feedback on writing samples, and seamlessly transition from one activity to another.

“Is ‘Mr. Holland’s Opus’ possible every time you walk into the class?” asked I.S. 3o3 Assistant Principal Monica Brady, one of the panelists, referring to the movie about the inspirational music teacher. “Is it going to happen every moment of that amazing class?”

Brady said teachers having their best days on the job would have trouble aspiring to some Danielson exemplars.

“There are things about it that are nutso,” she said. “Kids can’t have choice every time they walk into a room.”

2. “Fist-to-Five” and other strategies for student-led grouping don’t always work

As a result of Danielson’s unreasonably lofty standards, educators say it can sometimes be smarter not to reach for them at all. That’s tough for ambitious teachers like Danielle Lerro, the I.S. 303 English teacher on the panel. She studied the rubric and knew what it took to be rated “highly effective” for each component.

In the first video shown, Lerro asked students to show their level of understanding of text they had just read, using fingers on one hand, with a fist being a low level and five being high. She used “fist-to-five” and left it up to them to partner with each other based on shared comfort with the material, a practice that is a “highly effective” way to assess instruction in real time, according to Danielson.

But leaving the decision up to students can be “deceptive,” Brady said, especially in a classroom with many high-needs students who might throw up more fingers than they should.

“Fist-to-five is great, but you need to be able to select sometimes who you know is getting it and who’s not,” Brady said she told Lerro.

In the next video, taken several months later, Lerro listed students before the class who she knew would need extra help and worked with them on a similar activity.

“In an ideal world, [students grouping themselves] would happen all the time,” Lerro said. “But in the world we live in it can’t happen all the time.”

3. It’s all about the school

Classroom observations are yet another instance where policy can make sense in theory but fail in implementation. That was a takeaway from some who joined the event’s conversation online.

A teacher on Twitter said that what is constructive in some schools can be destructive and punitive in others.

Maisie McAdoo, an official with the city teachers union, agreed. “A lot depends on administrators getting this right and too many use the rubrics against teachers,” she responded.

Brady acknowledged the concern during the panel. She said observations are not a “gotcha” technique at I.S. 303, noting that she’ll sometimes cancel a planned observation if she realizes early on that students are especially unruly or the teacher is having an abnormally rough day. If she wanted to use Danielson against a teacher, she said she probably could.

“In the same way that it’s setting a standard, it’s also a standard that’s really easy to fudge,” she said. “It’s really easy to use to get someone.”

4. Observation by video can sometimes trump the real thing 

The limitations of using video as a proxy for an actual observer are easy to imagine. One is that video can miss a student’s reaction when a lesson starts to click for him. Another, New America’s Laura Bornfreund said, is that administrators who watch the video later on in their office might not be fully focused, or “guilty of multitasking.”

But Lerro said she recently picked up on an unanticipated advantage. Most administrators are respected authority figures in their schools. So when they pop into classrooms for observations, it often alters the behavior of the class, an externality that could mask a teacher’s weakness in classroom management.

Lerro said Brady was a good example. Students are always on their best behavior when they know Brady is near, Lerro said, so Brady gets the most realistic view into the classroom when she’s watching tape.

5. Educators want unlikely changes to their evaluation plans

When asked to name the one thing they want to see changed in the city’s teacher evaluation system, Brady and Lerro offered some revisions that might never come.

Brady said she worried about the pace at which the system was being imposed on schools. “For me it’s the speed,” Brady said. She said she was worried that the evaluations could breed distrust among teachers and administrators.

Lerro said that she had little confidence in calculations of teachers’ impact on students’ growth on state tests, a method known as “value-added.” Value-added scores calculated by the city starting in 2007 rarely matched up to a teacher’s actual quality, she said.

“They were all over the map based on what I know about good teaching,” Lerro said. “There was no correlation from what I saw.”

Her anecdotal experience matches what studies of the scores have verified. The city abandoned the scores two years ago, but state law requires that 20 percent of a teacher’s 2013-2014 rating be based on a value-added score determined by the state. Gov. Andrew Cuomo’s strident defense of the state’s evaluation law this year suggests that the formula is unlikely to be changed anytime soon.


How the Adams 14 school district is empowering parents to join the classroom

PHOTO: Yesenia Robles
A parent volunteer works with two kindergarteners on reading as part of a pilot program at Dupont Elementary School that is training parents to become paraprofessionals.

Raeann Javier would like to know what she can do to help her second-grader read better. Sometimes, sitting with her daughter, the best she could offer was, “You know how to do this.”

Javier, a single mother, also would love to land another job to earn more for her family.

A pilot program launched by Adams 14 School District in Commerce City may help her with both.

The school district is trying to build more knowledgeable, active parents through classes and volunteer time working with young students struggling to read. For those who are interested, the program also provides parents a path to become paraprofessionals, or teacher’s aides.

The initiative is one way the nearly 8,000-student suburban district — facing state intervention this year after years of poor academic performance — is trying to turn things around.

District surveys found parents were looking for ways to become more supportive.

Javier, one of 17 mothers in the program, said she already feels like she has become a more patient parent less than a month in. She also is interested in becoming a paraprofessional to supplement the income she earns as an at-home nurse.

“It’s a little bit tough. I make it work,” Javier said. “But this would really, really help.”

Other parents taking part in the pilot program already were volunteers at their kids’ schools.

“They usually just did the normal things like helping with copying or sorting papers,” said Jesse Martinez, Adams 14’s director for family and community engagement. “But we really wanted to change that dynamic. We wanted to pull in our parents to tap their potential and bring them in to support their children.”

One of the parent volunteers, Susana Torres, was an elementary school teacher for 10 years before coming to the United States. Now with three children in district schools, Torres jumped at the opportunity to get back into a classroom.

“This is my thing,” Torres said. “I love the program.”

Torres also helps other Spanish-speaking moms who are part of the program. She said that even though they don’t have the teaching background she does, the program has made it easy for all of them to learn to help kids. “All you need is a passion to make change,” she said.

Pat Almeida, the principal of Dupont Elementary, where the program is being piloted, said the goal is also to help more students become proficient in reading before third grade — especially those who are not far behind but just need a boost to get to grade level.

“We’re able to give them more repetition so they can apply that to their reading,” Almeida said. “If they’re able to have more repetition, their progress is going to be accelerated.”

Dupont Elementary is among the Adams 14 schools that is struggling, though the school isn’t yet facing sanctions like the district as a whole is this year.

District officials have been working on setting up reforms all year to present to the state as a suggestion for their corrective action, including getting help from an outside company for developing curriculum and testing. Increasing parental engagement through this and other new efforts, like having teachers visit families at home, are part of the work to improve the district.

The parent-to-para program is being funded with money from the Denver-based Rose Community Foundation (Rose also supports Chalkbeat) and Climb Higher Colorado, a coalition of advocacy groups that support strong academic standards and tests.

At Dupont, while the parent volunteers work with almost 75 students that they pull out of class for about an hour, teachers can spend the time in class working with students who need the most help.

An instructional coach supervises the moms to work with groups of two to six students and helps them plan lessons each day for kids.

During one lesson this week, parents were helping kindergarteners learn how to differentiate between capital and lowercase letters and how to sound out words. Some students were still having trouble identifying letters, while one boy was writing words so quickly he was standing up, moving around and at one point fell.

The volunteers said it’s rewarding to see the kids catching on.

“Knowing that just a little bit of our time can help them is a good feeling,” said volunteer Adelaida Guerrero. “It’s an excellent opportunity for them and for us.”

For Maria Rodriguez, the program has unexpectedly given her another benefit — bringing her closer to her teenage daughters. She said she joined the program because when a bilingual program for her two oldest daughters was removed seven years ago, she had stopped being able to help them on their school work.

When Rodriguez heard about the program, she thought she could prepare to help her younger children, a second and third grader, before they too required more help than she could offer.

“It’s brilliant,”Rodriguez said. “I’ve been helping them work on their vowels.”

Within the last week, the two older girls came to Rodriguez complaining that she hadn’t ever worked to help them in the same way, and asking to join in during the at-home lessons. Over time, the girls had kept their ability to speak Spanish, but never learned how to write it. Now they were asking to learn alongside their younger siblings.

“They have that apathy of adolescence that makes them not always want to get close to us as parents,” Rodriguez said, tearing up as she recalled the moment. “I honestly felt really good.”

the write way

What’s missing from the conversation about the state’s ditched literacy test for teachers?

PHOTO: Monica Disare
The Board of Regents applauds James Tallon at his last meeting, moments after officially voting to eliminate a literacy test for prospective teachers.

In a major change to teacher certification, New York officials decided prospective teachers will no longer have to pass an “academic literacy” test in order to enter classrooms.

It didn’t take long for media outlets to jump on the news, raising concerns that teachers who struggle to read and write could now be able to enter New York’s classrooms.

State officials say that argument is misguided. Since aspiring teachers must earn a college degree, and pass three other certification exams, they argue, illiterate applicants will not make the cut in the first place. The exam also is inherently flawed, they say, and kept a disproportionate number of black and Hispanic teachers out of schools.

But experts say that narrow debate about literacy misses a broader conversation. They argue that the test was never meant to protect against a flood of teachers unable to read and write. It was, however, intended to help ensure high teacher quality, they say, and the question is whether the current certification process furthers that goal.

The Academic Literacy Skills Test, which the state implemented a few years ago, was part of a larger movement to elevate the quality of the teaching profession. Officials thought at the time that a more rigorous test of reading and writing should be part of that mix.

But literacy tests for teachers got their start in a different era, said Linda Darling-Hammond, a leading national education researcher who now runs an education policy think tank. They were originally implemented in the 1980s when there were fewer hurdles to entering the teaching profession, she said.

“That might have made sense at that time in those places,” Darling-Hammond said, but added there is little evidence today that literacy tests are a good way to screen for effective teachers. There’s also no widespread concern, she said, that illiterate teachers are entering the profession.

“I think at this point, there is not strong evidence about that,” she said.

Ken Lindblom, dean of the School of Professional Development at Stony Brook University, who has taught prospective teachers, agreed that the focus on literacy is misplaced.

“It’s simply not the case that we have all these teachers … and they’re illiterate and we need to stop,” Lindblom said. “This is a false conundrum that we have invented.”

Dylan Roth, who is studying to become a teacher in a graduate program at Queens College, said he felt “insulted” by news coverage suggesting an epidemic of teacher illiteracy. “They speak of the ALST as if it were the line in the sand keeping horridly illiterate and unqualified teachers out of the classroom,” he said. “Yet besides brief mentions of teachers unions [in the articles], there is no virtually no input from teachers themselves who have gone through the process of certification,” he said.

Roth pointed out that the test’s fee (more than $100) poses a burden for aspiring teachers already paying for seminars, textbooks, tuition and more. Meanwhile, he said, the test is unnecessary when similar questions could simply be added to one of the three other certification tests, a proposal the state has offered.

Still, some say this conversation is about more than just a test — it’s about how the state can build a superior teacher workforce. Ian Rosenblum, executive director of the Education Trust-NY pointed to a 2007 study that found recruiting teachers with stronger certification status or SAT scores could improve student achievement.

“Research shows that having teachers with stronger academic skills makes a meaningful difference in student outcomes, and that is why we believed that maintaining the ALST … is important for equity,” Rosenblum said.

Daniel Weisberg, CEO at TNTP, an organization focused on creating more effective teachers, says the best way to attract high-quality teachers isn’t a tough literacy test. He thinks the emphasis should be on more teacher observation instead.

But he said, over the years, he has seen some prospective educators who want to become teachers even though they lack basic reading and writing skills. In order to avoid certifying those teachers, state officials could create a more narrow test of basic skills, he said.

“What you need is a surgical tool, not a chainsaw,” Weisberg said. “With a lot of these certification exams right now … they end up being a chainsaw, not a surgical tool.”

State Education Department officials plan to add a long reading and writing requirement to the Educating All Students Test, a change that is still being reviewed. They could not yet say whether a student who failed a new literacy portion of another exam could still become a teacher. According to a state education official, the test is expected to include the new literacy portions by January 2018.

The other certification tests already involve reading and writing. The edTPA, a performance-based assessment that asks students to videotape a lesson, requires them to write about their teaching practice. The exams require writing, but the edTPA handbook says the rubrics “do not address the quality of your writing,” and does not penalize test-takers for grammar and spelling errors, though it suggests the ability to effectively communicate is critical.

The other exams include content questions and some questions that require written responses. Supporters of eliminating the literacy test argue that even though prospective teachers are not given a writing score, literacy is embedded in the exams.

The edTPA “requires teacher candidates to organize their arguments, to logically sequence claims,” said Jamie Dangler, vice president for academics at United University Professions, which represents SUNY employees and who co-chaired the state’s edTPA task force. “That’s how you assess literacy. It’s the ability to write, but it’s more then that.”

Stephen Sigmund, executive director of High Achievement New York, a coalition of groups that promote rigorous standards, isn’t convinced. Literacy is crucial to teaching and should be assessed separately, he said. And while the state may include more literacy questions in a different exam, it’s a mistake to eliminate the test without having a fully developed alternative, he said.

“I don’t know enough about the specifics of the test but I take the [State Education] Department and the Regents’ word for it that they think there was a flawed test,” Sigmund said. “So fine, if there are problems with the test, fix the test.”