More than scores

Beyond test scores, Tennessee looks at how to evaluate districts and schools under ESSA

PHOTO: Grace Tatter
Assistant Education Commissioner Nakia Towns speaks during Wednesday's testing task force meeting, which largely focused on what measures besides test scores should be included in Tennessee's accountability plan under the Every Student Succeeds Act.

As Tennessee education leaders draft a new plan to evaluate the state’s schools and districts, they’re trying to move beyond test scores.

But exactly which metrics to incorporate into the state’s new accountability equation are still very much in the air as the State Department of Education develops an accountability plan in answer to the federal Every Student Succeeds Act, or ESSA.

Education Commissioner Candice McQueen put that challenge Wednesday to her 21-member testing task force of educators, politicians, and researchers — just one of several groups the state is consulting.

Passed by Congress last year, ESSA gives states more latitude in defining and measuring school and district success. While states still must report annual test scores, ESSA encourages choosing metrics that paint a fuller picture of what goes on inside schools.

“We do think we have a great moment and a great opportunity in Tennessee to really make a statement about our values,” Assistant Commissioner Nakia Towns told task force members. “What are our values beyond test data?”

Here are factors that could soon be part of Tennessee’s accountability plan, and what task force members are saying about them.

What do you think? Which metrics would best help the state determine which districts and schools that are serving schools well, and which need interventions? Let us know in the comments below.

Discipline data

No consensus emerged on whether discipline data, like expulsion and suspension rates, should be used to grade districts and schools. But department officials and task force members insisted the issue needs to be addressed, even if it’s not part of the accountability system.

Kyle Southern, policy director for the State Collaborative on Reforming Education (SCORE), cited a University of Pennsylvania report showing that Tennessee suspended and expelled more students than any other state in the South — and that students of color are disproportionately affected.

Towns said she has been looking at discipline stats from across the state, especially in her home district in Knoxville.

“Frankly, some of the disparities around which subgroups are being most affected by suspensions, it’s shocking,” she said. “This is something that has very severe consequences when you talk about school-to-prison pipeline, dropout rates.”

Strengths: Data on suspensions and expulsions — and racial disparities — is easily accessible. Holding schools accountable for high numbers of suspensions and expulsion incentivizes alternatives, which would increase the amount of time students spend in school. Helping districts focus on discipline could improve school culture and equity across the state, as well as improve emotional supports for students.

Drawbacks: Districts report suspensions and expulsions differently. There’s the potential unintended consequence of schools not enforcing rules and being lenient on disruptive or unsafe behavior. And some schools and districts have more resources, such as guidance counselors and social workers, to help support students.

Chronic Absenteeism

Task force members were most adamant about including chronic absenteeism in the reboot of Tennessee’s accountability system, although they weren’t sure what the definition of chronic absenteeism should be.

Weakley County Schools Director Randy Frazier said students’ performance starts suffering from absences long before the courts consider them truant. “We know it’s an issue that affects instruction,” he said.

Pros: Research consistently shows a correlation between attendance and post-secondary success. Also, absenteeism gives insight into school culture and relationships in ways that other metrics can’t.

Cons: Absenteeism can be outside of schools’ control, and not all schools have access to support services that help ensure kids show up to schools.

A student social-emotional and learning survey

Tennessee districts already administer surveys around school climate to students and teachers. Department leaders asked task force members if districts and schools would want to be evaluated based on the results of a student survey about social and emotional supports, such as relationships with teachers and skills such as perseverance. Members answered that social and emotional learning conditions are important, but questioned how reliable a survey would be.

Pros: Social and emotional learning is linked not only to improved academic outcomes, but to improved conditions long after school, such as employment and happiness. It’s an untapped way to assess the non-academic factors at a school, such as relationships, that educators know are important.

Cons: Younger students might not understand the survey, and it might take a long time to develop one that can accurately capture both what a school does and how it feels for its students.

Early post-secondary options

About half of task force members said that early post-secondary measures — such as access to AP or IB courses and community college classes — should somehow be included in the accountability system. Towns listed a lot of questions to dig into if the state uses post-secondary opportunities as a metric: What counts as an early post-secondary opportunity? Should all opportunity be counted the same? And, since courses vary across schools and districts, how should performance in such courses be evaluated?

Pros: Expanding early post-secondary options aligns with the state’s Drive to 55 initiative to increase the number of students who finish some sort of postsecondary education.  Performing well in college prep or vocational courses is a strong indicator of success in college or careers. And being able to rack up college credits in high school saves students money.

Cons: Questions are plentiful about what “post-secondary” opportunities look like in elementary and middle schools, and what metrics could be used to measure them. Also, some schools can’t afford to offer many college preparatory or vocational courses, either because they don’t have enough students, lack funding, or both. Task force members want to avoid tracking wealthier students into different classes and opportunities than their less-affluent peers.

Access to highly effective teachers

Last fall, a State Department of Education report found that low-achieving students are less likely to have a teacher that scored well on their state teacher evaluation than their high-achieving peers — even though low-achieving students have more to gain from strong teaching. The department has earmarked access to quality teachers as a major equity issue.

Pros: Research shows that teachers have a significant impact on student performance, perhaps more than any other factor schools can control.

Cons: While there’s a consensus that effective teachers are important, not everyone agrees what a highly effective teacher is, and how that effectiveness can be measured. Some districts have an advantage in attracting good teachers, either because of the pay they can offer or because of geography: School systems near cities simply have a greater supply of teachers to choose from than rural districts.

An education U-turn

Carmen Fariña wants to help New York City teachers get better at teaching. But some of her own reforms are getting in the way

PHOTO: Christina Veiga
The math team at P.S. 294 in the Bronx discuss a recent lesson during the 80 minutes of professional development time carved out by the city's most recent contract with the teachers union.

It was a Monday afternoon and school was out at P.S. 294. But there was plenty of learning happening inside the blue-and-yellow building in the Bronx.

Teams of teachers were gathered in classrooms on almost every floor. One group discussed a recent math lesson on how to identify patterns; another analyzed which questions had stumped students during recent statewide tests. A third was thinking about new ways to encourage discussion in the classroom.

In each huddle, they were learning a valuable lesson from each other: how to become better teachers.

What’s happening at P.S. 294 is what Carmen Fariña envisioned when she became chancellor of the country’s largest school system. Among the veteran educator’s most deeply held beliefs is that school improvement starts in the classroom — by helping teachers get better at teaching.

“To me, everything that happens in the classroom is the most crucial thing in the building,” Fariña told Chalkbeat.

Many of Fariña’s reforms reflect that vision, including the city’s contract with the teachers union, which carves out time for professional development each week. But another set of changes Fariña made — overhauling the education department bureaucracy — has sometimes worked at cross purposes, taking power away from those who know schools best.

Strapped superintendents and staffers sidelined in support centers now oversee much of the training teachers encounter. Fariña herself acknowledges it has sometimes been a struggle to meet the diverse needs of schools under the new system.

One Bronx principal said he sees that struggle firsthand.

“What some people call ‘supporting instruction with professional development,’ other people would call ‘bloated bureaucracy,’” the principal told Chalkbeat. “I have no interest in their professional development, and they don’t know my school.”

***

Like much of what has happened at the education department under Mayor Bill de Blasio, the chancellor’s emphasis on teaching the teachers marks a radical shift from the preceding administration.

Former Mayor Michael Bloomberg and Chancellor Joel Klein wanted great teachers in every classroom, too. But their position was that it was easier to hire top talent than cultivate it. Instead of pouring resources into teacher improvement, they set about measuring teachers to weed out those who were ineffective.

“Joel didn’t believe in professional development at all,” said Eric Nadelstern, who served as deputy chancellor for school support and instruction under Klein. “His question was, ‘Is it easier to change the teacher — or to change the teacher?’” Klein himself did not agree to be interviewed for this story.

When Fariña took the helm, educators took heart that one of them was in charge again. With 50 years of experience in New York City classrooms, she was the first chancellor in more than a decade who didn’t need a waiver, which the state requires when a school leader does not have the experience set by law for the job.

“When de Blasio named Fariña chancellor, it was a message,” said Norm Fruchter, a researcher at New York University who previously served as a de Blasio appointee to the Panel for Educational Policy. “The pendulum was going to shift back towards valuing instruction.”

In one of her first moves as chancellor, Fariña helped hammer out a contract with the United Federation of Teachers, the union that had clashed for years with Bloomberg and Klein. Among its most significant changes: giving teachers 80 minutes after school every Monday to work on improving their craft. The contract also created new leadership positions that gave extra pay to skilled teachers who agreed to take on coaching roles in their schools.

Taken together, those moves helped create a structure for helping teachers improve within their own schools.

“The thing with the most value in schools is time,” said Phil Weinberg, deputy chancellor of the department’s Division of Teaching and Learning. “The biggest thing that we’ve done is to honor the fact that learning has to happen by creating time.”

***

In the education world, there is much debate around whether professional learning really works. Plenty of research suggests that typical models do not. Educators have their own disparaging vocabulary to describe those models: drop-and-go, spray-and-pray, even drive-by professional development. The idea is that one-off lectures and workshops are rarely effective in changing teacher practice, let alone improving how much students are learning.

However, recent research suggests there are ways to get it right. A review of 35 different studies, released in June by the Learning Policy Institute, found common themes in professional learning programs that actually improve student performance. Those programs provide coaching, are collaborative and typically happen on the job — much like what’s happening at P.S. 294.

P.S. 294 The Walton Avenue School serves students who are traditionally among the city’s lowest-performing — those who are homeless, learning English, or have disabilities. Yet it outperforms the city average on standardized tests.

PHOTO: Christina Veiga
Daniel Russo (center) working with the math team at P.S. 294

The school has taken on Algebra for All, a de Blasio initiative that helps schools change the way they teach math. P.S. 294 also has teacher leaders paid to share their knowledge with teams of their colleagues. Those teams then work together in the 80 minutes each week reserved for professional development. All of that comes together under a principal, Daniel Russo, who makes sure teachers get the feedback they need to improve their practice.

“We come back every couple of months and say, ‘How are we doing on this? What fell by the wayside and what are ways that we can do better?’” Russo said. “Everyone is going to contribute to, and benefit from, the greater knowledge that there is in the room.”

For all its ambitions, the 80 minutes don’t always work as planned. In about a dozen interviews with teachers and principals, many school staff said they appreciate that the Monday sessions have provided time and space to think about their practice. But others said that time can feel wasted or forced.

“Everyone is very busy at our school, and that’s just another meeting that has to take place to plan more meetings,” a Bronx high school teacher told Chalkbeat. (The teacher, like many educators interviewed for this story, agreed to speak only on the condition of anonymity.)

“A lot of times we’re not really sure what we’re going to do on a given day,” the teacher said. ”It’s not very focused throughout the year.”

***

Why, then, are some schools making good use of the new training time and at others, teachers feel like it’s being frittered away?

One factor: changes to the way principals are supervised and how schools get support.

Under Bloomberg and Klein, principals who needed help turned to dozens of “networks” scattered throughout the city. Principals opted into networks based on their schools’ needs, regardless of where the school or network were located in the city. The network providers were expected to solve problems for schools, or principals could vote with their feet and join different networks.

"That’s the system-wide idea of support now: taking people away from kids."David Baiz, former principal of Global Technology Preparatory

As chancellor, Fariña took a different approach. She promptly rebuilt the department’s Division of Teaching and Learning, which had been dissolved after she left the DOE in 2007. Once again, there was an office at the Department of Education’s central headquarters dedicated to actively helping schools decide what and how to teach.

She also empowered superintendents, calling them the “instructional leaders” of their districts, and upped the years of experience required to land the job. They evaluate principals but are also responsible for making sure schools get the support they need.

In the place of networks, Fariña opened “field support centers,” which serve hundreds of schools but don’t hold supervisory power. Unlike networks, most centers only work with schools located in the same borough. Superintendents and support centers are expected to work together to help schools improve teaching.

Crucially, that doesn’t always happen. The result can work against the 80 minutes, by distancing decision-making about professional development from schools — and complicating it, too.

Our principal is “held with her hands behind her back,” said Corey Taylor, a music teacher at P.S. 33 in the Bronx. “She has to do what she’s being told by her higher-ups.”

Now, principals are expected to ask their superintendents for help, who then turn to field support centers. Some principals and support centers do work directly together, though Weinberg said that’s not the preferred system.

“The ideal thing is that you’re in constant conversation with your superintendent,” he said. “It would be hard for each borough field support center to hear 145 different requests every day, from each of their schools.”

PHOTO: Christina Veiga
Teachers at a training for Computer Science for All, a citywide initiative

Relationships between superintendents and support centers don’t always run smoothly, and both are tasked with overseeing many schools. Superintendents have staffs of around six people, yet may be responsible for dozens of schools. Support centers work with up to 323 schools, with an average caseload just below 200.

With superintendents acting as a filter between schools and support centers, many principals report a divide between what they’re offered and what they want to learn.

“There’s a disconnect between the reality of what’s going on in classrooms, and the offerings,” one Manhattan principal told Chalkbeat. “It usually comes down to: Teachers need to learn, very specifically, techniques, tips, philosophies that affect their own practice.”

When they work well, support centers might send staff to a school to provide targeted help requested by its principal. But, faced with heavy caseloads, the centers often respond to schools’ needs by creating borough-wide professional development sessions that can vary in quality. In the city’s most recent survey of principals, only 73 percent said they were satisfied with the support they get from the centers.

One Manhattan teacher said she went to sessions offered by the support center last year and was disappointed with what she found. The presenters led a lesson on “guided reading,” a technique that includes introducing vocabulary and breaking students into groups, but they seemed fuzzy on how to execute the practice in the classroom.

“Teachers were actually correcting them,” the teacher said. “They’re removed and they forget what it’s like to be a teacher.”

***

Despite Fariña’s emphasis on classroom-based learning, many of the support centers’ professional development sessions are happening outside schools, while class is in session. At three separate support centers, almost all the trainings for teachers offered during the month of May were held during school hours.

"We’ve set aside the time. We’ve set up the space. Can we just manage it?"Michael Mulgrew, president of the teachers union

That wouldn’t have happened under Bloomberg, according to Nadelstern, the former Klein deputy. He said his policy was that teachers and principals should not be pulled away from schools while students are in the building.

“That’s the system-wide idea of support now: taking people away from kids,” said David Baiz, the former principal of Global Technology Preparatory in East Harlem. “That’s not really the best way that adults learn: to sit in a meeting away from the context of their work environment and then try to come back and incorporate it.”

In addition to out-of-office professional development, superintendents host monthly meetings, pulling principals out of their schools for the entire day. In some cases, they include meals paid for by vendors who present professional development sessions based on educational products they’re selling.

“There’s just this feeling among almost every principal that I know,” a Bronx principal told Chalkbeat. “Like meeting after meeting after meeting and requirement after requirement are being added, and really drowning out the time needed for real collaboration.”

In an interview with Chalkbeat, Fariña admitted that professional development run by outside vendors is “not that effective.” She also acknowledged there have been growing pains as the superintendents and field support centers try to meet the needs of all the schools they serve.

“It’s been more of a struggle in some places where there was a more diverse need,” she said.

PHOTO: Christina Veiga
Schools chancellor Carmen Fariña looks over a card from students in her office.

To address that, Fariña said the centers have been working on “modules” based on different areas of need. A module may highlight effective strategies for teaching students who are learning English, for instance, and come with a series of professional development courses that can be run over a period of multiple weeks.

“Each principal can adapt it as they see fit,” Fariña said.

***

Weinberg said it is easy, in a system as large as New York City, to point to “random” weak points.

“What our real goal is, is continuous improvement,” he said. “I think that we make mistakes, oftentimes, by looking at one anecdotal example as a way of disproving a larger movement.”

Michael Mulgrew, president of the teachers union, said the department needs to pay closer attention to how schools are using the resources that are now available. While the the 80 minutes of professional development time is a game-changer, he said, it can also vary in usefulness depending on school culture, principal leadership and how well superintendents and the field support centers can provide help.

“We’ve set aside the time. We’ve set up the space. Can we just manage it?” Mulgrew said. “The fact that the chancellor made this a priority when she came in is the reason why you see the school system moving forward. My fear is, have we reached a plateau?”

It may be tricky, however, to balance the kind of oversight that Mulgrew envisions with the personalization that teachers and principals say is necessary for effective professional development. But the city is evaluating its own work to make sure it’s hitting the mark for teachers and schools.

“Teaching is really fascinating and difficult work,” Weinberg said. “We need to approach this hard job with the humility that says we have the ability to learn more — and we want to learn more.”

Chalkbeat reporters Monica Disare and Alex Zimmerman contributed to this report.

Share your thoughts on the quality of New York’s professional development for educators in our short survey.  

past deadline

State lawmakers end session without passing mayoral control. Where does that leave us?

PHOTO: Geoff Decker
Speaker Carl Heastie on the Assembly floor.

The final day of New York state’s scheduled legislative session has come and gone — but there’s no final deal on mayoral control of city schools.

Lawmakers wrapped up the legislative session late Wednesday, though they could return to address mayoral control of New York City schools at a later date. The provision expires at the end of June, but blowing the session’s deadline takes state officials one step closer to letting the mayor’s governance of the nation’s largest school system lapse.

“This evening, the state legislature will adjourn its 2017 legislative session,” Senate Majority Leader John Flanagan said in a statement. “We would have preferred to have tied everything up with a nice neat bow and returned to our districts with nothing at all left on our plate, but under the circumstances, that just wasn’t possible.”

He also suggested that he supports mayoral control of city schools, as long as it comes with help for charter schools. “I support the renewal of mayoral control, as do my Senate Republican colleagues,” Flanagan said.

Meanwhile, Assembly Speaker Carl Heastie admitted this outcome wasn’t perfect. “Sometimes in politics you don’t always get what you want,” he said on Wednesday night. He also reportedly said he had “no intention of coming back.”

For the past two years, Mayor Bill de Blasio has secured only one-year extensions of the policy, thanks largely to his fraught relationships with Senate Republicans and Governor Andrew Cuomo. (Former Mayor Michael Bloomberg received a seven-, then a six-year extension.)

Without mayoral control, New York City schools would revert back to a system with 32 community school boards — something even Mayor Bill de Blasio’s opponents do not support. Yet lawmakers are stuck in a deadlock over whether the extension should come with concessions, such as eliminating New York City’s charter school cap.

Mayoral control has lapsed before, providing a blueprint for what it might look like if it happened again. In July 2009, under Bloomberg, its expiration caused a brief reconstitution of the city’s Board of Education. But it took only a month before lawmakers returned to Albany and passed a multi-year extension.

The relatively small disruption caused by the lapse in 2009 leads some observers to conclude that letting the law expire will bring little harm to schools, teachers or students. That’s a far cry from the “chaos, gridlock, and corruption” predicted by New York City Schools Chancellor Carmen Fariña.

But the first scenario assumes the law will be reinstated as quickly as it was in 2009, said Tim Kremer, executive director of the New York State School Boards Association. Every year is different, and this one is marked by “deep-seated” policy and personality conflicts between the mayor and Albany lawmakers, Kremer said. Also, notably, the Senate was in the midst of a leadership crisis when lawmakers failed to renew mayoral control in 2009.

“I think people are taking false comfort in saying ‘Listen, we blew through the deadline last time and nothing happened; we can do that again,’” Kremer said. “They really are playing a little bit with fire.”

So what exactly would it look like if mayoral control lapsed? Chalkbeat spelled that out in a step-by-step guide back in 2009, informed in part by a memo sent by Bloomberg’s staff outlining how they saw the transition at the time.

First, city officials would have to reconvene a citywide Board of Education with five appointments made by borough presidents and two by the mayor. That board would have the power to select a chancellor. The city followed that script in 2009, which resulted in a unanimous vote to retain Joel Klein as chancellor.

Some observers, like David Bloomfield, a professor at Brooklyn College and the CUNY Graduate Center, feel confident the same would happen this time around.

“I fully expect that to go without any problem and that they will proceed to appoint Carmen [Fariña],” Bloomfield said.

To test his theory, Chalkbeat reached out to all five borough presidents earlier on Wednesday. Officials from three offices responded. The Manhattan Borough President Gale Brewer’s director of communications said she is committed to keeping Carmen Fariña as chancellor. Officials from Borough President Eric Adams’s office said he was focused on renewing mayoral control. Officials from Queens Borough President Melinda Katz’s office did not commit to keeping the chancellor.

“The only commitment Borough President Katz will make at this time is to appoint a representative to the reconstituted Board of Education,” officials wrote in an email.

If mayoral control lapsed for more than a month, New York City would head into uncharted territory. At some point, the city is required to revive the community school boards, but those elections wouldn’t be held until spring 2018.

That leaves months of limbo. In 2009, there was some discussion of whether the chancellor could appoint trustees to community school boards in the interim. But a spokeswoman for Mayor Bill de Blasio said he interprets the law to mean there would be no community school boards until the following May. That means no community input, no ability to appoint a permanent superintendent, and likely no rezoning votes, she said.

Even though there’s only a slim chance this fight will last until May, de Blasio said he doesn’t want to take any chances.

“When you open up Pandora’s box,” the mayor said at a press conference Wednesday, “you don’t know what happens next.”