study says...

What the F? Here are the surprising ways failing grades helped New York City schools

PHOTO: Getty Images

Sometimes a bad grade is a good sign.

New research suggests that in New York City, giving schools a failing or near-failing grade reduced teacher turnover and helped attract more effective educators.

These results cut against the theory that low district-assigned grades would have precisely the opposite effect. They also provide new evidence that the now defunct A–F grading scale sparked improvements at underperforming city schools.

The study will be published in the peer-reviewed Journal of Human Resources. Because it focuses on comparing low-rated schools to those that received slightly higher grades, the study doesn’t offer a definitive verdict on how the grading system the city used from 2007 to 2014 affected the district as a whole.

What the findings do show: Schools that received a letter grade of D or F saw a 20 percent reduction in teacher turnover, compared to similar schools that barely scored one grade higher. The schools with lower grades also attracted higher-performing teachers, as measured by their ability to improve student test scores over time.

“This is an important counterpoint to the people who worry about the stigmatization effects of letter grades,” said Jonah Rockoff an education researcher at Columbia University, who reviewed the journal article and has studied the city’s school grading system.

And the results dovetail with Rockoff’s own previously published research showing New York City schools that received low grades went on to boost student achievement in reading and math.

Grades were based on student improvement on test scores over time, absolute student performance, and the school environment as measured by attendance and surveys. The grading system was at the heart of a sweeping set of reforms Mayor Michael Bloomberg’s administration implemented to give schools more autonomy, but also harsh consequences if they did not measure up.

Principals at top-scoring schools could become eligible for bonuses and extra funding, while bottom-ranked schools were subject to leadership changes or even closure. Critics feared that low grades would stigmatize schools and demoralize educators. And they voiced concerns that progress scores bounced around too much from year to year to be reliable. Echoing some of those worries, schools Chancellor Carmen Fariña, who retired in April, eliminated the system in 2014.

To measure the grading system’s impact during its first two years, the study uses data from 156 elementary and middle schools and compares those that barely received a D or F with those that scored just high enough to earn one grade higher. Lower ratings reduced teacher turnover, from about 11.5 percent of teachers leaving each year to 9 percent. That goes against both conventional wisdom and previous research from Florida, which also put in place letter grades for schools.

Reduced turnover could also signal that educators at low-performing schools had more trouble than their counterparts getting hired elsewhere. But Rebecca Dizon-Ross, the paper’s author and a professor at the University of Chicago, argues that it’s more likely that schools responded to the low grade by making improvements that encouraged teachers to stay.

Her study points out that teachers who attempted to leave low-rated schools were no less likely to be hired at other city schools, and lower-rated schools didn’t see fewer teachers transfer into them. Moreover, the teachers who did join schools after they received a failing grade tended to be more effective at improving test scores than teachers hired at top-ranked schools.

In addition, teachers at low-scoring schools tended to give their principals higher ratings after the poor grades became public — suggesting that school leaders made improvements in light of the added pressure from the district.

“If you’re at a school with a good principal that is responding well to a low accountability grade, that’s where teachers wind up staying,” Dizon-Ross said.

But the new findings come with some important caveats.

For one, each school’s letter grades were published after the school year started, making it difficult for teachers to leave immediately or mid-year, and potentially dampening any stigma effect.

And there is some evidence that not all bottom-ranked schools reduced turnover. Dizon-Ross’ data shows schools that received the lowest scores in the D or F range actually had higher turnover than schools that also received a D or F but were on the cusp of a higher grade. Her method focuses on schools that just missed the cutoff, so it can’t completely account for the experiences of schools that were the lowest performers.

Dizon-Ross acknowledged those limitations, but said her findings still show that the letter-grading system produced some important benefits, at least in New York City.

“The letter grading policy seemed to have this upside — it was designed to encourage low-performing schools to improve,” she said. “It’s a hopeful picture for accountability.”

Frequently asked

New Denver teacher contract: We answer the most common questions about the tentative pact

PHOTO: Joe Amon/The Denver Post
Students in class at Dora Moore ECE-8 during the second day of the Denver Public Schools teachers strike.

One reason many Denver educators didn’t like the district’s old ProComp pay system was that it was too complicated and unpredictable. Both sides agree that the deal reached early Thursday morning creates a much simpler pay system for teachers.

But educators — and the general public — still have a lot of questions about the tentative ProComp agreement, which still needs to be ratified by union members and the Denver school board. Here we’ve answered some of the most common questions we’ve heard since the end of the strike.

How do I place myself on the salary schedule?

The salary schedule is made up of “steps” and “lanes.” The “steps” represent years of service for which a teacher had a positive evaluation. The “lanes” represent levels of education. The new schedule has 20 steps and seven lanes.

Worked in Denver Public Schools for five years and have a master’s degree? Go to step five and then slide your finger over to the master’s degree lane. That’s your base salary.

Did you have a year when your evaluation wasn’t good? Go back one step. Have an additional 18 credits on top of your master’s degree? Go up one more lane.

Teachers can also go up a lane once they hit the 10-year mark because the district wanted to reward longevity. Other milestones that merit a lane change: earning national board certification or an advanced license, or completing six “professional development unit” training courses.

Still not sure? Denver Public Schools plans to put a salary calculator on its website soon.

What if I have more than 20 years of experience?

If you have 20 or more years of experience, you’re placed at the top of the salary schedule, on step 20. After step 20, you’ll get yearly cost-of-living raises. You’re still eligible to change lanes, but you won’t get any more step raises.

Does the district know everything it needs to know about individual educators to pay them the correct salary?

Denver Public Schools plans to send letters or emails this spring to every teacher and special service provider (nurses, counselors, and others) covered by the contract, laying out where the district believes that employee falls on the schedule based on information they have on file. Educators will have a certain amount of time to correct any wrong information and get on the correct step and lane for the 2019-20 school year.

Under the new salary schedule, it looks like I’ll earn less next year than I do now. Am I taking a pay cut?

No. The agreement includes a “hold harmless” clause that ensures everyone will get a raise next year. Those whose salaries are higher now than they would be under the new schedule will get a cost-of-living raise each year until the salary schedule catches up with them.

How are bonuses and incentives different under the new contract?

The bonuses and incentives are different in three ways: There are fewer of them, the dollar amounts are different, and the dollar amounts won’t change year to year.

This year, there are six bonuses and incentives offered by the district: one for educators who work in Title I schools where 60 percent or more of the student population qualifies for subsidized meals; one for educators who work in hard-to-fill positions; one for educators who work in “hard-to-serve” schools; one for educators who work in one of 30 “highest-priority” schools; one for educators who return year over year to those schools; and one for educators who work in schools deemed top-performing or high-growth, as based on school ratings.

Here’s what’s left in the new contract: Teachers in Title 1 schools and those in hard-to-fill positions, such as secondary math, will get $2,000 a year. Teachers who return year over year to 30 highest-priority schools will get $3,000 a year. Teachers in 10 schools deemed “distinguished” will get $750 a year, with the criteria to be determined by the district and the union.

Why aren’t the district and the union tying bonuses to test scores anymore?

Unions have traditionally been skeptical of paying teachers based on student test scores because the scores are so closely correlated with factors like race and household income. In Denver, these bonuses were also less predictable for teachers because the district often changed the criteria it used to rate schools and award “top-performing” bonuses.

The district also came to see these bonuses as canceling out the effects of bonuses for teachers at high-poverty schools. A teacher could get nearly the same kind of monetary reward by moving to a more affluent school or by staying in one where students face more challenges. The new bonus system provides clearer monetary benefits to working in a high-poverty school.

Why did the union agree to keep the incentive for highest-priority schools, when that had been such a sticking point?

In any negotiation, there’s give and take and a lot of moving pieces. 

Here’s what lead negotiator Rob Gould said to district officials during bargaining: “We are open to the incentive because we know it’s important to you. And we’re willing to entertain your ideas if we can get the base salary schedule that our teachers need. Because if we can get the base salaries we need, we can keep our teachers in Denver.”

This was also an issue that divided teachers, with some teachers at schools that received the highest-priority incentive pushing to keep them.

Did teachers get a better deal out of the strike than the district’s last offer before the strike?

Teachers were getting a raise no matter what. The district was offering an average 10 percent raise before the strike (this included a cost-of-living raise that was agreed to back in 2017). Now teachers will get an average 11.7 percent raise, though individual teachers will see a wide range.

The district is putting the same amount of new money — $23.5 million — into teacher compensation as it was offering before the strike. It can give a larger average raise with that same amount of money because the incentives are smaller than under the previous proposal and because of limits on how teachers can use training to get raises. That gives the district more predictability about how many teachers will get raises each year.

Union leaders call the deal a win. They secured more opportunities for teachers to earn raises and move into higher categories on the salary schedule, including through completing training partially during work hours at no additional cost. And teachers can get to $100,000 in 20 years, rather than the 30 years in the last district proposal.

However, individual teachers aren’t necessarily getting more base pay next year than they would have under the district’s last offer. Early-career teachers without advanced degrees would have earned more in base pay under the district’s last offer. The teachers who do better under the deal reached after the strike are veteran educators with more education.

To take two examples: A second-year educator with a bachelor’s degree and no extra credits or training would have earned $47,550 in base pay under the district’s last offer before the strike but will earn $46,869 under the deal reached this week.

But a 20-year educator who has a master’s degree and an advanced license who has been with the district for 10 years will earn $88,907 in base pay under the new agreement, compared with $87,550 under the district’s last proposal before the strike.

The union fought for this kind of salary schedule in part to address a longstanding complaint that teachers have little reason to stay in a district where base pay levels off.

You can see the salary schedule from the district’s last offer here and the schedule from the tentative agreement here.

Is this deal financially sustainable for the district?

Denver Public Schools Chief Financial Officer Mark Ferrandino says that is the “million-dollar question,” perhaps closer to the “half-billion-dollar question,” since that is roughly how much the district spends on educator compensation.

Ferrandino believes the answer is yes, with the standard caveat that all projections are just that.

What will be cut to pay for this?

The district plans to cut $20 million from administrative costs over the next two years. That includes cutting 150 jobs in the central office and ending all executive bonuses. The bulk of it — $13 million — will go to fund the ProComp agreement.

District officials have not yet said which central office jobs will be cut, though Superintendent Susana Cordova has said cuts will be to “discretionary” departments. Departments that will not be cut include special education, English language acquisition, and transportation, she said.

Teachers will get a raise. What about paraprofessionals, bus drivers, custodians, and cafeteria workers?

These other district employees, much lower paid than teachers, are not covered by the contract that was the subject of the strike. Cordova has said these workers also deserve raises and a portion of administrative cuts will go to pay for them.

But how much of a raise will they get? That will all be worked out over the next few months and include discussions with the unions that represent these employees.

Will striking teachers get back pay?

Not according to district officials. After this story was published Friday, we asked for further clarification on this. We received this statement Saturday morning:

Superintendent Cordova understands that when teachers make the choice to strike, they are doing so to make a statement and bring attention to the importance of the issue at hand. Foregoing pay during the time that a teacher is not working is a challenging decision that no one makes lightly, and consequently, brings with it an impact that is intended to push for change.

DPS did not feel that it would be fair or appropriate to provide back pay to striking teachers when many others — including more than 40 percent of classroom teachers — chose to remain at work this week. However, DPS is working with the DCTA to offer all teachers the opportunity to attend a Saturday session to replace the professional development day that was cancelled in the days leading up to the strike. Any teacher who attends will be paid a day’s salary.

When will the new agreement go into effect? How long will it last?

Assuming both sides ratify it, the new agreement technically (and retroactively) went into effect Jan. 19, the day after the old one expired. But educators won’t start receiving the new salaries, incentives, and bonuses negotiated under it until Aug. 1. The agreement expires Aug. 31, 2022.

Teens Talk Back

‘Mr. Mayor, we cannot afford to wait.’ Teen group says New York City diversity plan doesn’t move fast enough.

PHOTO: Courtesy/Teens Take Charge
Teens Take Charge members at a "virtual" press conference in New York City on Thursday

A teen group representing students from more than 30 New York City high schools sharply criticized a recent report from Mayor Bill de Blasio’s School Diversity Advisory Group as offering no real solutions for increasing integration in the city’s starkly segregated high schools.

At a virtual press conference on Thursday, broadcast live on Facebook by Teens Take Charge, students expressed support for the report’s broad policy aim of achieving greater integration but also disappointment that the findings offered few specifics for how to reach this goal. The mayor’s Diversity Advisory Group has said a follow-up report will provide more details later this year.

“We have been told to wait, to be patient, that change is coming soon,” said Tiffani Torres, a junior at Pace High School in Manhattan. “Mr. Mayor, we cannot afford to wait any longer.”

Teens Take Charge has long advocated for greater efforts to end segregated enrollment patterns in the city’s high schools. Sokhnadiarra Ndiaye, a junior at Brooklyn College Academy High School, said that students’ expectations of the mayor included his announcing “a comprehensive plan” — even if it took years to realize — “to racially, socioeconomically, and academically integrate high schools before the end of this school year,” she said.

Among Teens Take Charge’s specific recommendations are doing away with academic screens for admission to the city’s high schools, a more transparent process for applying to them, and more resources for low-income schools. Early last year, the group produced an Enrollment Equity Plan for increasing educational opportunities for low-income black and Hispanic students.

And because concrete plans for increasing integration would take time, Ndiaye said the teen organization supports several interim measures as well to address inequities in the school system. These include providing more college and career counseling for junior and seniors at low-income, under-resourced high schools. The teen group would also like to see the city provide vouchers to low-income families to access extra-curricular activities and programs offered by private companies or the ability to participate in such programs at other public schools if theirs don’t offer them. (Some city teens joined a class-action lawsuit against the education department and Public School Athletic League for allegedly denying black and Hispanic students equal opportunity to play on school sports teams, in violation of local human rights law.)

Torres described how Teens Take Charge has had “several meetings and phone conversations with Department of Education officials over the past year,” and schools chancellor Richard Carranza has stated that students have his ear. “We’re listening,” he tweeted in response to a Chalkbeat story with excerpts of the students’ views.

In December, the city’s education department posted a new job listing for a “Student Voice Manager” who would gather students’ thoughts on education policies. But while acknowledging this seat at the table, several students expressed frustration at the slow pace of change.

Bill de Blasio’s office declined to comment about Teens Take Charge’s concerns or their specific recommendations, beyond referencing remarks the mayor already made about the School Diversity Advisory Group report.

Doug Cohen, an education department spokesman, said in a statement, “We’ve taken real steps toward school integration,” pointing to initiatives such as a $2 million diversity grant program for school districts and communities citywide to develop their own local diversity plans, and a program that enables middle-schoolers to visit college campuses. “We know there is more work to do, and we thank Teens Take Charge for its continued advocacy on these issues,” he added.

Students at the group’s event urged swift change. “They know our plan; they have our information,” said Sophie Mode, a sophomore at Brooklyn Millennium. “They need to take action now.”