progress reports

New York

For most students, no benefit to a school's F grade, study finds

A study examining whether getting poor grades on city progress reports prompted schools to improve their students' test scores found little evidence of such a boost. The study, released today by the conservative-leaning Manhattan Institute, asked the question by comparing schools with progress report raw scores that were roughly the same, but just different enough to get different letter grades. In fact the two groups showed about the same amount of progress — except in fifth-grade math, where students in failing schools made "significant and substantial improvement" compared to their peers in schools that had been assigned a grade of D, according to the study. The progress reports assign letter grades to schools based primarily on improvements in students' test scores. Since the first reports were released a year ago, the program has been the subject of sustained criticism: Parents and teachers have complained about unfair stigmatization of good schools, and statisticians have charged that the reports are driven as much by error as by actual school improvement. The study's architect, Manhattan Institute senior fellow Marcus Winters, called his findings "mixed-positive" in favor of the progress reports. Those findings were the subject this morning of a panel discussion sponsored by the Manhattan Institute featuring Winters, Columbia University economist Jonah Rockoff, and two officials from the Department of Education's accountability office, including its CEO, James Liebman.
New York

Improvement in progress report grades: real or random?

Last year, the first round of progress reports attracted anger and ridicule. Perhaps because far fewer schools received low grades, the response this year has been more muted, making room for measured, evidence-based discussion of the DOE's methodology in constructing the reports. Over at Eduwonkette, Harvard education professor Daniel Koretz offers a lengthy critique of the progress report methodology. He notes that test scores alone are not a legitimate way to evaluate schools; New York State's tests were not designed to be used in "value-added" analysis like that behind the progress reports; and the progress reports, like all accountability systems, place pressure on school administrators that likely leads to score inflation. In addition, he writes that the DOE's formula does not take into account "interval scaling," or the reality that different amounts of "value" are required to move students from one proficiency level to the next at different points on the proficiency spectrum. (In June, I wrote about how interval scaling might contribute to the finding that No Child Left Behind has helped high-performing students less than their low-performing peers.) But those problems exist in many test-based, value-added accountability systems — Koretz writes that New York's progress report system has its own set of errors. The tremendous variation in schools' grades from last year to this year probably has less to do with school improvement than sampling and measurement error, he writes. Here's an illustration of the effect of error. I first calculated the variation in schools' grades between last year and this year and then graphed it against their enrollments.
New York

80 percent of schools land top grades on DOE's progress reports

As early reports suggested they would, Mayor Bloomberg and Chancellor Klein announced today that more than half of all elementary and middle schools received higher progress report grades this year than last year, the first that the reports were issued. In all, 79 percent of schools earned As or Bs, more than 20 percentage points higher than last year. (High school reports will come out later this fall, after data about August graduation and Regents performance is taken into account.) "We're just as proud of the F student who became a C student as we are of the A student," Bloomberg said, contrasting the city's accountability system against the state's, which condemns low-performing schools even if they are on the upswing. The DOE's press release is filled with impressive statistics about schools' performance on the reports. The reports released today are meant to highlight student progress, as opposed to raw performance, which the state uses to judge schools. Sixty percent of a school's grade is based on "progress," or how much individual students improved or fell behind in the last year. Schools also get "extra credit" if students with special needs — such as disabilities, English language learner status, or poor performance in the past — do particularly well. Raw student performance does make up 25 percent of a school's grade, and the results of the Learning Environment Surveys that parents and teachers took this spring make up the remaining 15 percent of the score. On each measure, schools are compared both to all city schools and to schools in their "peer group," made up of schools that have similar demographics. Bloomberg said the reports, which are available online, make school performance transparent and help administrators and teachers to focus their attention and resources on the students who need it most. Finally, he said, the reports are the only measure where schools are held accountable for improving student performance — and accountability, he emphasized, breeds success, with schools earning higher marks this year even though progress report grades were issued last November for the first time, just two months before state English tests and four months before state math tests. The DOE's chief accountability officer, Jim Liebman, who spearheaded the progress report initiative, cited a recent paper by Columbia economist Jonah Rockoff that concludes that new accountability systems can produce real effects in a very short time. Administrators at PS 5, the Bedford-Stuyvesant elementary school where the press conference took place, said their progress report grade last year pushed them to help their students more.
New York

At PS 8, families cry foul on year's first progress report grade

With their schools' 2007-2008 progress report grades due out next week, principals are likely to spend their weekend planning either a victory celebration or damage control. PS 8 in Brooklyn Heights At PS 8 in Brooklyn Heights, families are trying to figure out what to think about their school's failing grade, especially because it earned a C last year and accolades this summer from Chancellor Klein, who held a conference at the school to announce that the school would expand to meet community demand, the Times reports today. Since the arrival of the current principal, Seth Phillips, in 2003, families in the zone have increasingly decided to stay put and enroll at PS 8 once their children reach school age. But according to the DOE's progress report formula, upper-grade students' test scores did not improve as much last year as they might have (and did at other schools), even though a majority of them scored at grade level or higher on state math and reading tests. Asked about the chancellor's July comments, DOE spokesman David Cantor told the Times, “Now that he has additional information about the school, his view has changed. The most important things about a school are student progress and performance, and in those areas this school isn’t measuring up.” Cantor also said parents and teachers noted "significant concerns" when responding to last year's Learning Environment Survey — but those concerns aren't apparent in the composite survey results, which put PS 8 in the top half of schools citywide in three of the four main categories and well above average on the fourth, "engagement."