study says...

What the F? Here are the surprising ways failing grades helped New York City schools

PHOTO: Getty Images

Sometimes a bad grade is a good sign.

New research suggests that in New York City, giving schools a failing or near-failing grade reduced teacher turnover and helped attract more effective educators.

These results cut against the theory that low district-assigned grades would have precisely the opposite effect. They also provide new evidence that the now defunct A–F grading scale sparked improvements at underperforming city schools.

The study will be published in the peer-reviewed Journal of Human Resources. Because it focuses on comparing low-rated schools to those that received slightly higher grades, the study doesn’t offer a definitive verdict on how the grading system the city used from 2007 to 2014 affected the district as a whole.

What the findings do show: Schools that received a letter grade of D or F saw a 20 percent reduction in teacher turnover, compared to similar schools that barely scored one grade higher. The schools with lower grades also attracted higher-performing teachers, as measured by their ability to improve student test scores over time.

“This is an important counterpoint to the people who worry about the stigmatization effects of letter grades,” said Jonah Rockoff an education researcher at Columbia University, who reviewed the journal article and has studied the city’s school grading system.

And the results dovetail with Rockoff’s own previously published research showing New York City schools that received low grades went on to boost student achievement in reading and math.

Grades were based on student improvement on test scores over time, absolute student performance, and the school environment as measured by attendance and surveys. The grading system was at the heart of a sweeping set of reforms Mayor Michael Bloomberg’s administration implemented to give schools more autonomy, but also harsh consequences if they did not measure up.

Principals at top-scoring schools could become eligible for bonuses and extra funding, while bottom-ranked schools were subject to leadership changes or even closure. Critics feared that low grades would stigmatize schools and demoralize educators. And they voiced concerns that progress scores bounced around too much from year to year to be reliable. Echoing some of those worries, schools Chancellor Carmen Fariña, who retired in April, eliminated the system in 2014.

To measure the grading system’s impact during its first two years, the study uses data from 156 elementary and middle schools and compares those that barely received a D or F with those that scored just high enough to earn one grade higher. Lower ratings reduced teacher turnover, from about 11.5 percent of teachers leaving each year to 9 percent. That goes against both conventional wisdom and previous research from Florida, which also put in place letter grades for schools.

Reduced turnover could also signal that educators at low-performing schools had more trouble than their counterparts getting hired elsewhere. But Rebecca Dizon-Ross, the paper’s author and a professor at the University of Chicago, argues that it’s more likely that schools responded to the low grade by making improvements that encouraged teachers to stay.

Her study points out that teachers who attempted to leave low-rated schools were no less likely to be hired at other city schools, and lower-rated schools didn’t see fewer teachers transfer into them. Moreover, the teachers who did join schools after they received a failing grade tended to be more effective at improving test scores than teachers hired at top-ranked schools.

In addition, teachers at low-scoring schools tended to give their principals higher ratings after the poor grades became public — suggesting that school leaders made improvements in light of the added pressure from the district.

“If you’re at a school with a good principal that is responding well to a low accountability grade, that’s where teachers wind up staying,” Dizon-Ross said.

But the new findings come with some important caveats.

For one, each school’s letter grades were published after the school year started, making it difficult for teachers to leave immediately or mid-year, and potentially dampening any stigma effect.

And there is some evidence that not all bottom-ranked schools reduced turnover. Dizon-Ross’ data shows schools that received the lowest scores in the D or F range actually had higher turnover than schools that also received a D or F but were on the cusp of a higher grade. Her method focuses on schools that just missed the cutoff, so it can’t completely account for the experiences of schools that were the lowest performers.

Dizon-Ross acknowledged those limitations, but said her findings still show that the letter-grading system produced some important benefits, at least in New York City.

“The letter grading policy seemed to have this upside — it was designed to encourage low-performing schools to improve,” she said. “It’s a hopeful picture for accountability.”

Follow the ratings

Illinois education officials laud their school ratings — but critics say they don’t go far enough

Illinois rolled out its new school accountability system in the Illinois Report Card late last month.

State education officials publicly lauded their new school rating system Friday, even as a new, nationwide analysis of school improvement plans criticized Illinois’ approach as too hands-off.  

While the state developed a clear rating system as the federal government requires, Illinois falls short in follow-through, according to the report from the Collaborative for Student Success, a non-profit advocacy group, and HCM Strategies, a policy analysis group.  

“The state is taking too limited a role in leading or supporting school improvement efforts,” said the report, which examined how 17 states are implementing school improvement plans under the federal Every Student Succeeds Act, which was passed in 2015 and replaced the No Child Left Behind Act.

Both those federal laws task states with identifying and helping improve underperforming schools and with creating criteria to judge which schools are doing well. Illinois rolled out its new school accountability system in the Illinois Report Card late last month.

State officials disagree with the criticism.

“Illinois is being held up as a model for other states to follow,” said Ralph Grimm, chief education officer of Illinois, speaking at the monthly state board of education meeting on Friday. “The entire (state) team has to be commended for providing useful information.”

Illinois’ rating system places every public school in the state into one of four categories based in part on scores on the annual PARCC standardized tests (click here to see how Chicago schools ranked).

Only about a third of Illinois students scored proficient or higher on PARCC tests administered last spring. In reading, 37 percent of students in grades 3 through 8 met that benchmark, while in math 31 percent did. Despite that, the state awarded 80 percent of its schools a “commendable” or “exemplary” rating. 

The state labeled 20 percent of schools “underperforming” or “low performing,” the only designations that could trigger state action. Intervention measures include improvement plans, visits from specialists, and additional funding.

The state released its ratings just days after Chicago released its own batch of school ratings, which take into account a different set of metrics and a different standardized test.

Grimm said the next step will be asking the state’s lowest-performing schools to draft improvement plans and then connecting them with experts to implement their changes.

The state ratings pay particular attention to how schools educate certain groups of students — such as children of color and English language learners. Improvement plans will focus on ways to raise their achievement levels.

Under the latest state rankings, nearly half of Chicago schools failed to meet the state’s threshold for performance, with a disproportionate number of high schools on the low-performance list. Nearly all of under- and low-performing Chicago high schools are on the South Side and sit in or border on the city’s poorest census tracts.

The state could grant underperforming schools $15,000, and  the lowest performers can apply for $100,000 under its IL-Empower program — which helped schools improve by funneling federal funds to them. Advocates have welcomed the change to a carrot to help schools pull themselves up, after years of sticks that overhauled and cut funding for low-performing schools.

Nationally, the Collaborative for Student Success report applauded Colorado for its streamlined application system, and Nevada for asking districts to directly address equity.

The collaborative criticized Illinois for failing to involve parents and community members in its plan. The group also said the state needs to give districts more guidance on putting together school improvement plans. 

carry on

These 16 Denver charter schools won renewal from the school board

PHOTO: AAron Ontiveroz/The Denver Post
Sebastian Cruz waves to Rev. Leon Kelly as he works with children in a classroom during his after-school program at Wyatt Academy in September 2018.

The Denver school board on Thursday night unanimously renewed agreements with 16 of the district’s charter schools. The lengths of those renewals, however, varied from one year to five years — and signaled the board’s confidence in the schools to deliver a quality education.

The board also accepted Roots Elementary’s decision to close and surrender its charter at the end of this school year. The Park Hill school is facing low enrollment and high costs.

Denver Public Schools is a charter-friendly school district that has for years shared tax revenue and school buildings with its 60 publicly funded, independently operated charter schools. The schools are controversial, though, with opponents viewing them as privatizing public education.

Every charter school in Denver has an agreement with the district that spells out how long it’s allowed to operate. To continue running after that time period, the charter school must seek renewal. The arrangement is part of the deal for charters: They get the flexibility to operate independently, but they must periodically prove to the district that they’re doing a good job.

The school board relies on one set of recommendations from Denver Public Schools staff and a second set of recommendations from a districtwide parent committee in deciding how long a leash to give each charter school. The district staff and the parents on the committee consider factors such as test scores, school culture, financial viability, and the strength of a school’s leaders when making their recommendations.

They also consider a school’s rating on Denver Public Schools’ color-coded scale based largely on academic factors. The School Performance Framework, or SPF, labels schools either blue, green, yellow, orange, or red. Blue means a school has a distinguished academic record, while red means a school is not meeting the district’s expectations.

The staff recommended the school board renew the charters of all 16 schools that applied. Two other charter schools — DSST: Cole Middle School and Compass Academy — are also up for renewal this year. But because they earned the district’s lowest rating, they must go through a separate process in which they will present a detailed improvement plan. Their renewals will depend on the strength of their plans, which is why they weren’t included in this batch.

The board approved the 16 renewals Thursday without discussion. All of the new terms begin next school year. Here’s the rundown:

STRIVE Prep Federal, a middle school in southwest Denver
Year opened: 2006
School rating: Green
Renewal: Five years

DSST: Green Valley Ranch High School, a high school in far northeast Denver
Year opened: 2011
School rating: Green
Renewal: Five years

Rocky Mountain Prep Creekside, an elementary school in southeast Denver
Year opened: 2012
School rating: Green
Renewal: Five years

DSST: College View High School, a high school in southwest Denver
Year opened: 2015
School rating: Green
Renewal: Three years, with a possible two-year extension

KIPP Northeast Denver Leadership Academy, a high school in far northeast Denver
Year opened: 2015
School rating: Blue
Renewal: Three years, with a possible two-year extension

KIPP Northeast Elementary School, an elementary school in far northeast Denver
Year opened: 2015
School rating: Yellow
Renewal: Two years, with a possible three-year extension

Rocky Mountain Prep Southwest, an elementary school in southwest Denver
Year opened: 2015
School rating: Yellow
Renewal: Two years, with a possible three-year extension

Wyatt Academy, an elementary school in northeast Denver
Year opened: 2003
School rating: Yellow
Renewal: Two years, with a possible two-year extension

KIPP Northeast Denver Middle School, a middle school in far northeast Denver
Year opened: 2011
School rating: Yellow
Renewal: Two years, with a possible two-year extension

Downtown Denver Expeditionary School, an elementary school in central Denver
Year opened: 2013
School rating: Yellow
Renewal: Two years, with a possible two-year extension

Denver Justice High School, an alternative high school for at-risk students in central Denver
Year opened: 2009
School rating: Yellow
Renewal: Two years, with a possible one-year extension

REACH Charter School, an elementary school in central Denver
Year opened: 2015
School rating: Yellow
Renewal: Two years, with a possible one-year extension

Monarch Montessori, an elementary school in far northeast Denver
Year opened: 2012
School rating: Orange
Renewal: One year, with a possible two-year extension

STRIVE Prep SMART, a high school in southwest Denver
Year opened: 2012
School rating: Orange
Renewal: One year, with a possible two-year extension

Academy of Urban Learning, an alternative high school for at-risk students in northwest Denver
Year opened: 2005
School rating: Red
Renewal: One year, with a possible one-year extension

Rise Up Community School, an alternative high school for at-risk students in northeast Denver
Year opened: 2015
School rating: Red
Renewal: One year