measuring up

Denver Public Schools lowering the bar on school rating measure in response to concerns

PHOTO: Andy Cross/Denver Post
Denver Public Schools Superintendent Tom Boasberg talks to sixth graders in 2012 .

Denver Public Schools is making a last-minute change to how it calculates its color-coded school ratings for this academic year by lowering the bar on one key measure.

The adjustment — which involves scores on state standardized tests in math and English — comes after school leaders were given preliminary ratings last week and raised concerns.

Those ratings, obtained by Chalkbeat, show that before the change DPS is adopting, about 45 percent of schools would have dropped at least one color on DPS’s five-color scale. About 38 percent would have stayed the same, while about 17 percent would have gone up. The district’s color ratings range from a high of “blue” to a low of “red.”

DPS officials have long known students’ scores on new, more rigorous math and English tests were likely to be lower than on previous state tests. But officials were not planning on tweaking the rating system as a result, reasoning that the new tests are more in line with what students are expected to know and lowering the bar would paint a misleading picture.

But DPS Superintendent Tom Boasberg told Chalkbeat on Wednesday that district officials changed their minds after seeing how the ratings shook out.

“When you talk about a framework in the absence of data, your discussion is going to be a lot less rich than when you have real data that makes you aware of issues that, on a more abstract and theoretical level, you’re not aware of,” he said.

The change relates to students’ status scores, meaning how many students met or exceeded expectations on state tests, known as PARCC. Status scores are one ingredient that goes into a school’s rating. Growth scores — or how much students learn year-to-year compared to their academic peers — are another and count for more.

Because the way the state calculates growth has stayed the same despite the switch to the new tests, those scores haven’t been impacted by the tests’ increased rigor the way status scores have. That’s why the district is focusing on the latter, Boasberg said.

Here’s what DPS is proposing to do and why:

In the past, the percentage of students who had to meet or exceed expectations for a school to earn a high DPS rating ranged by grade level and subject between 20 and 50 percent, Boasberg said. The preliminary ratings released last week used those same cut points.

Once school principals saw the ratings, Boasberg said they raised concerns about using different cut points for different grade levels and subjects. While doing so made sense for the old state tests, on which younger students tended to do better than older students, he said it makes less sense for the new, more consistent PARCC tests.

And while DPS’s rating system has long been tougher than the state system, Boasberg indicated some principals wondered whether it was too tough. For example, he said, they told the district it didn’t seem fair that a school where, say, 40 percent of third-graders met or exceeded expectations on the state English test could get a low DPS rating when an average of only 37 percent of Colorado third-graders met that bar.

As a result, the district is planning to recalculate its ratings using cut points that range from 20 percent to 40 percent, instead of 50 percent, Boasberg said. However, in a letter to principals Wednesday obtained by Chalkbeat, he warned the cut points for all tests would rise to 50 percent in 2018.

The district is aiming to provide recalculated ratings to school leaders by Oct. 21 and release them publicly on Oct. 27. Boasberg said he expects “very few” schools’ ratings to change. Even so, he said, “the consistency and coherence of the (rating system) is important. We didn’t feel it was right to ignore thoughtful and well-grounded concerns about that … even if it means extremely little difference for most schools.”

At a school board meeting Monday, before district staff decided to lower the cut points, board members said they worried how families would react to drops in color ratings.

“Let’s say you had a school that’s been green for a couple of years,” said board president Anne Rowe. “This comes out and that school is orange. As a parent, I might say, ‘Whoa! What happened to my school?’ My question is, ‘How do we communicate: what does that mean?’”

Research shows families rely on the rating system, known as the School Performance Framework, when making choices about where to send their children. And because of Denver’s universal school choice system, students can request to enroll in any school in the city. Since school budgets are based on enrollment, their choices have financial consequences.

“I’m really worried about public perception, where a lot of people just look at that color and make decisions about schools,” said board member Mike Johnson.

Boasberg told the board it will be important to caution families about the quirks of this year’s ratings. Those cautions include that because last year was just the second year Colorado students took PARCC, only one year of growth data is available.

In the past, DPS has used two years’ worth of growth data to calculate schools’ ratings. The district believes using more data smooths out one-time anomalies that can cause scores to swing up or down, he said.

Since growth counts more than status in DPS’s rating system, having just one year of data means schools are likely to see bigger swings this year, Boasberg explained.

This year’s ratings are especially high-stakes because the district plans to use them to help make decisions about whether to close persistently low-performing schools under a new DPS policy called the School Performance Compact.

The policy uses three criteria to determine whether to close a school. The first is whether that school ranks in the bottom 5 percent of all schools based on multiple years of ratings.

The board is currently scheduled to vote in December on whether to close schools.

testing talk

‘Virtually meaningless’ or ‘steady progress’? New York City reacts to this year’s state test scores

PHOTO: Grace Tatter

English and math exam pass rates inched up in New York City this year compared to last year — more than they did in the state as a whole, city officials announced Tuesday.

The annual release of test scores created a wave of reactions from education stakeholders across the state. Charter school advocates claimed victory, the state teachers union called them “meaningless” and Mayor Bill de Blasio said they represent the “painstaking work” of schools across New York City.

Here is a sample of reactions:

The mayor touted his own education agenda.

“These improvements over the past four years represent painstaking work – student by student, classroom by classroom, and school by school. It’s steady progress towards a stronger and fairer system for all. We are focused on building on these gains and others – such as the highest-ever high school graduation rate – to deliver equity and excellence for every public school student across the city, no matter their zip code.” — Mayor Bill de Blasio

Charter advocates said it shows the strength of their approach.

“New York City charter public schools are continuing to show us poverty is not destiny in the greatest city in the world. Charter public schools offer the promise of closing the achievement gap and today’s results show they are delivering on that promise. It’s been almost 20 years since New York passed its charter law and these public schools are now out of the experimentation phase: not only should their lessons have more reach, but so should they.” — StudentsFirstNY Executive Director Jenny Sedlis

Success Academy highlighted its push for more school space.

“These results should inspire the de Blasio administration to immediately support Success Academy and other high-performing charters to serve more students in public space.” — Eva Moskowitz, Success Academy founder and CEO

The state’s teachers union called the test scores “virtually meaningless.”

“They don’t count for students or teachers — and they shouldn’t count. They are derived from a broken testing system; are rooted in standards that are no longer being taught; and — for now — are the foundation of a totally discredited teacher evaluation system. The test-and-punish era damaged the trust and confidence of parents in our public education system, as evidenced by the continuing strength of the opt-out movement, and we believe dramatic changes are needed to win them back.” — NYSUT President Andy Pallotta

The city’s teachers union said they represented “progress.”

“Thanks to the efforts of teachers and other staff members across the city, our students are making solid, sustainable progress and the nation’s largest school system is moving in the right direction.” — UFT President Michael Mulgrew

Other groups took the chance to criticize opt-out.

“The results show the right thing to do is to keep moving forward, not tear down high standards and end annual assessments like opponents call for. The continued rise in proficiency scores is a clear sign that high standards are preparing students for future challenges, and parents are increasingly rejecting misguided calls to ‘opt out’ of the state’s annual check-ups. Both of these are good trends for every student in New York, no matter where they are growing up.” — High Achievement New York Executive Director Stephen Sigmund

And some pushed for more dramatic change.

“While we are pleased to see the test scores move in the right direction for New York City students overall, we are concerned about the persistent gaps that exist for students with disabilities and English language learners. Teaching students to read is one of the most fundamental tasks of schools.  With only 5.6% of English language learners and 10.7% of students with disabilities scoring proficiently in reading, the city must do more to support these students and ensure that they receive high-quality, evidence-based instruction that targets their individual needs.”— Advocates for Children Executive Director Kim Sweet

Testing Time

New York’s state test scores are coming out today. Here’s what we’ll be watching for.

PHOTO: Dylan Peers McCoy

At long last, New York state English and math test scores are being released today, according to state officials.

Though the tests have been criticized as providing only a snapshot of what students have learned, they are still one of the main tools used to judge the progress of schools, students, and major education initiatives.

Because the tests themselves held steady between 2016 and 2017, the test scores will provide a brief glimpse into whether students are making progress. That could also mean smaller changes than last year, when English scores shot up nearly eight percentage points.

But stable comparisons will only make a temporary appearance in New York education. Next year, the state has announced it will shorten testing by two days, which will no doubt call yearly comparisons into question again.

Here are a few storylines we’ll be watching as the state prepares to release test results:

Is the city’s approach to education policy working? What about signature programs like “Renewal”?

When test results jumped almost eight points in English and roughly one point in math last year, Mayor Bill de Blasio was quick to say they showed “pure hard evidence” his policies were working.

The state’s top education policymakers, however, cautioned that due to changes in the test, an “apples-to-apples” comparison to the year before was impossible. The changes included offering students unlimited time and shortening the exams slightly. This year, state tests were kept consistent in order to make those comparisons possible.

That raises an important question: Without changes to tests, will the results still be good news for Mayor Bill de Blasio and Chancellor Carmen Fariña? It is particularly important for high-profile efforts like the city’s “Renewal” turnaround program, which is entering the year in which the mayor said it should show results.

The only other year under de Blasio’s tenure when the state had fairly consistent testing compared to the prior year was 2015. In that year, test scores inched up one point in math and two points in English.

It may be a good thing if there are only small increases again, said Jennifer Jennings, an associate professor at NYU who has studied testing.

“We hope that they’re gradually inching upwards,” Jennings said. “Very large swings are often evidence that something is off.”

What will happen to opt-out?

For the last two years, about one in five students across New York state have boycotted state tests in protest. That number is significantly lower in New York City — though it has been growing. In 2016, 2.4 percent of city students sat out of the English exams and 2.8 opted out of math in New York City.

The opt-out rate acts as a litmus test of the public’s reaction to state education policy. The movement started in response to a series of state reforms, including adoption of the Common Core and test-based teacher evaluations.

Despite changes the state made last year to appease families upset about the tests, opt-out rates remained relatively consistent. (In fact, they ticked up a bit.) This year, the state has embarked on a process to reshape learning standards and submit a new plan to evaluate schools under the new federal education law.

Will that be enough to defuse some of the tension? Early results indicated opt-out rates may have decreased statewide, but the final tally will likely be released today with state test scores, as it has in past years.

What about equity, which is at the heart of the city’s agenda?

Each year, test results show a disheartening fact: Certain subgroups, such as black and Hispanic students, fare worse than their peers on tests. English learners and students with disabilities also historically score below average.

City and state officials have placed equity at the heart of their agenda. Mayor Bill de Blasio’s “Equity and Excellence” initiatives and his approach to turning around struggling schools are predicated on the idea that schools — particularly those in low-income communities — need resources in order to be successful. State officials have put equity at the center of their plan to reshape education policy under the Every Student Succeeds Act.

De Blasio’s critics, however, have argued the best way to address equity issues is tackling segregation in New York City schools. (The mayor released a preliminary diversity plan, but has been fairly slow to endorse integration as a strategy for school improvement.)

So, are the extra resources helping to close the gap between students of color and their white and Asian peers? This year’s test scores will help sort out that question, though many of the mayor’s major initiatives could take years to come to fruition.