grading on a curve

Teachers give new Regents exam scoring system mixed reviews

The brand-new library at Evander Childs opened so teachers from other schools could grade Regents exams there.

Last year, the Evander Childs Campus got a new library, replete with rows of new computers and a mural depicting scholarly pursuits.

The library opened its doors for the first time last month — but not to students. Instead, it housed teachers from other high school campuses, who convened there to try out a new model for grading students’ final exams.

Regents exams, which students must pass to graduate from high school, have been scored by the teachers who administered them since the Regents exam program began in the nineteenth century. But mounting concerns about cheating — spurred on by the finding that students hit the minimum passing score at a disproportionately high rate — have prompted the city and state to make changes to how the exams are graded.

The state’s test security overhaul calls for schools to stop grading their own Regents exams by June 2013. The changes are meant to reduce opportunities and incentives for teachers to inflate their students’ scores, which under state law could factor into teachers’ evaluations in the future. The shift would bring Regents exam grading in line with how most states score high-stakes exams and with New York State’s requirements about elementary and middle schools’ exams.

Buoyed by its own concerns about cheating and softer forms of score inflation, the city has sped that timeline up. In January, a handful of schools tested out a system to ensure that teachers do not grade their own students’ exams.

Department of Education officials expanded that system, known as “distributed scoring,” to more than 160 schools this spring.  Most of the schools deployed teachers to centralized locations such as Evander Childs, and teachers from 17 schools tested a system for grading exams online. In total, about 107,000 exams were graded under distributed scoring last month.

Teachers who participated in the pilot gave it mixed reviews. Some said the system made them better graders because they considered only the answers, not the students, when assigning scores. But others said the system of musical graders was complicated, time-consuming, and likely to lead to unfairly deflated scores. And a small number of missing tests highlight the potential cost of logistical mishaps.

Department of Education officials solicited feedback from teachers who piloted the new system and said they would use that information to improve it before the next round of exams in 2013. Shael Polakow-Suransky, the department’s chief academic officer, said the pilot included a wide range of schools from across different districts and networks to elicit as full a range of feedback as possible.

“It was just trying to get a right mix so that we could actually see where the challenges will be and where we need to make adjustments for next year,” he said.

Concerns about fairness

For Richard Mangone, a retired social studies teacher recruited to grade U.S. History exams at the Prospect Heights Campus, few changes are needed. He said the scoring process at his site was the most efficient he had seen in 30 years of grading exams.

He also said he found it easier to grade fairly. One finding that prompted the city’s February audit was that teachers issued a disproportionate number of 65s — the lowest passing score — on Regents exams, suggesting that they might be bumping up the scores of students on the verge of passing. That’s less likely to happen now that teachers are not grading their own students’ tests, Mangone said.

“It’s not that you’re less objective, but it’s easier,” Mangone said about distributed scoring. “You’re just looking at the response.”

Some said have argued the bulge of 65’s reflects not padded scores but concern for students most at risk of failing. At a panel last August, high school social studies teacher (and GothamSchools Community section contributor) Stephen Lazar said that when teachers are more invested in their success, they are more attentive while grading, preventing careless scoring errors from costing a student the score he needs to graduate.

Monica Mazzocchi, who teaches at New Utrecht High School, which was a scoring center, said she prefers grading her own exams for the same reasons.

“Because it’s not their students, will they care as much as we care?” she said about other graders.

Other teachers said the lack of context could be problematic for other reasons. Peter Lapré, a social studies teacher at Park East High School, said he teaches his students extensively about the Venetian salt trade, even though the subject is not covered in standard course materials. This year, his students’ exams were graded online by teachers at other schools — some of whom might not be familiar with that topic.

“I’m concerned my students who chose to write about that were graded unfairly because the teacher didn’t know that information,” he said.

A teacher from Harry S. Truman High School said she worried that other graders wouldn’t be aware that students from her school take U.S. history in ninth grade rather than in their junior year, as students in most schools do, and would grade them according to the standards they would apply to high school juniors.

Fears of score deflation

The Truman teacher, who scored exams at DeWitt Clinton High School and asked not to be named because she feared repercussions, said she found that her concerns were warranted: Her school’s test scores in the subjects graded at the central location dropped significantly, even though they rose in subjects graded in the old model. And Lapré said his scores were stable from last year, even though his school had doubled the time devoted to global studies instruction in an effort to boost scores.

Both teachers said they thought the new system placed more pressure on teachers to grade harshly by exposing them to oversight from their colleagues and supervisors. The Truman teacher said that because no one at her site wanted to be seen as too lax, teachers debating between two scores usually tended to round down instead of up.

Plus, each grader was assigned a three-digit identification number and assigned to write them next to every response he or she scored.

“You’re well aware you’re being watched,” Lapré said.

Each exam was also marked with the student’s name and school. Arthur Goldstein, a social studies teacher at Francis Lewis High School, which did not participate in the pilot, said he was concerned that information could bias graders against students.

“I wonder if a bunch of papers go to a closing school [to be graded], if they won’t look at it and make my kids pay for it because we’re a good school,” he said.

Another teacher whose school did not participate in the pilot said he worried that bias could cut the other way, disadvantaging students whose names or schools suggested they were likely to be black or Hispanic because teachers would expect them to perform less well.

The city’s progress report system for evaluating schools judges high schools in large part by their Regents exam pass rates, and rates that fall from one year to the next would result in a lower grade. The system also weighs each school’s performance against that of other schools with similar students. When schools’ annual letter grades are announced this fall, some schools that used distributed scoring will have been compared to schools whose exams were scored under the old system.

Department officials said the distributed grading model actually shields students from unfairness. Before teachers even began grading tests at the centralized sites, they completed an exercise to make sure they shared an understanding about what makes an essay worth one score rather than another. First, each teacher graded the same essay, and then members of each grading discussed their rationales before comparing their assessments to the state’s guidelines.

The actual scoring happened in committees of four or six, with two teachers grading every essay. Discrepancies of more than one point between the teachers’ scores would trigger another reading by a third teacher, according to a department official who worked on the new system. That rarely happened, teachers who participated in the pilot said.

A large-scale logistical undertaking

Test scores weren’t the only things that moved as a result of the pilot: The physical tests also had to be transported around the city, posing a logistical challenge. In one extreme hiccup, 17 exams taken at Franklin D. Roosevelt High School in Brooklyn that were supposed to be taken to New Utrecht for grading were lost.

The Department of Education’s Office of Special Investigation is looking into what happened and whether distributed scoring played a role, according to Marge Feinberg, a department spokeswoman. She said the exams, which were taken by students in FDR’s evening school for students at risk of dropping out, are the only ones missing from June’s Regents period.

“We are working with school staff to find the exams,” Feinberg said.

Other logistical issues had graders concerned about distributed scoring’s efficiency. The teacher from Truman said she and her colleagues spent long stretches in DeWitt Clinton’s library, doing nothing while they waited for exams to be collected or distributed.

A Brooklyn teacher who commuted with 24 colleagues to score at New Utrecht said he said he found it inefficient to commute each day to a school other than his own. And he worried that the presence of 100 extra people in the building while New Utrecht’s students were taking final exams was disruptive.

But the teacher, who asked not to be named because he feared repercussions, said he saw a value in handing off his students’ exams for others to grade.

“What I would recommend personally is, give me Utrecht’s, I’ll give them my papers, and we can stay in our own buildings,” the teacher said.

Summer remix

Ten stories you may have missed this summer (and should read now as the new school year kicks in)

PHOTO: Caroline Bauman
Gabrielle Colburn, 7, adds her artistic flair to a mural in downtown Memphis in conjunction with the XQ Super Schools bus tour in June.

Labor Day used to signal the end of summer break and the return to school. That’s no longer the case in Tennessee, but the long holiday is a good time to catch up on all that happened over the summer. Here are 10 stories to get you up to speed on K-12 education in Tennessee and its largest school district.

TNReady is back — with a new test maker.

Last school year ended on a cliffhanger, with the State Department of Education canceling its end-of-year tests for grades 3-8 in the spring and firing testmaker Measurement Inc. after a series of missteps. In July, Commissioner Candice McQueen announced that Minnesota-based Questar will pick up where Measurement Inc. left off. She also outlined the state’s game plan for standardized tests in the coming year.

But fallout over the state’s failed TNReady test in 2015-16 will be felt for years.

The one-year void in standardized test scores has hit Tennessee at the heart of its accountability system, leaving the state digging for other ways to assess whether all of its students are improving.

Speaking of accountability, Tennessee also is updating that plan under a new federal education law.

The state Department of Education has been working with educators, policymakers and community members on new ways to evaluate schools in answer to the federal Every Student Succeeds Act, or ESSA, which requires states to judge schools by non-academic measures as well as test scores.

Meanwhile, issues of race and policing have educators talking about how to foster conversations about social justice in school.

In the wake of police-related killings that rocked the nation, five Memphis teachers talked about how they tackle difficult conversations about race all year long.

School closures made headlines again in Memphis — with more closings likely.

Closing schools has become an annual event as Tennessee’s largest district loses students and funding, and this year was no exception. The shuttering of Carver and Northside high schools brought the total number of district-run school closures to at least 21 since 2012. And more are likely. This month, Shelby County Schools is scheduled to release a facilities analysis that should set the stage for future closures. Superintendent Dorsey Hopson has said the district needs to shed as many as two dozen schools — and 27,000 seats — over the next four years. A Chalkbeat analysis identifies 25 schools at risk.

Exacerbating the challenges of shifting enrollment, families in Foote Homes scrambled to register their children for school as Memphis’ last public housing project prepared to close this month amid a delay in delivering housing vouchers to move elsewhere.

The new school year has officially begun, with the budget approved not a moment too soon for Shelby County Schools.

District leaders that began the budget season facing an $86 million shortfall eventually convinced county commissioners to significantly increase local funding, while also pulling some money from the school system’s reserve funds. The result is a $959 million budget that gives most of the district’s teachers a 3 percent raise and restores funding for positions deemed critical for continued academic progress.

The district also unveiled its first annual report on its growing sector of charter schools.

With charter schools now firmly entrenched in Memphis’ educational landscape, a Shelby County Schools analysis shows a mixed bag of performance, while calling on traditional and charter schools to learn from each other and promising better ways to track quality.

summer mix tape

Ten stories you might have missed over the summer (and should read now as a new school year begins)

PHOTO: Nicholas Garcia
Stand for Children received funding to support parent education.

There is no such thing as time off from covering education. While school doors were shuttered, plenty happened this summer on the Colorado education beat. Here, we’ve compiled stories that we hope prove useful as you ease back into your fall routines.

We’ve got your immunization data right here … 

For the second year, Chalkbeat tracked down immunization data for more than 1,200 schools in Colorado’s largest school districts. Our database revealed that Boulder remains a hotspot for the anti-vaccination movement, students in districts with racial and income diversity are more likely to get their shots and nearly half of schools in the database did a better job this year tracking students’ immunization records. Read our news story about the findings, check out these six charts that dig into the numbers and search for school-level data here.

Denver Superintendent Tom Boasberg reflects on his sabbatical (a break not everyone appreciated)

In June, Denver Public Schools’ longtime schools chief returned from a six-month unpaid sabbatical in South America with his family. “It made us appreciate the extraordinary resources we have here,” he said in an interview about his experience.

A milestone for Colorado charter schools on diversity, but not so much on integration

For the first time, Colorado’s charter schools educated a larger proportion of racial and ethnic minorities than district-run schools, a state report showed. We took a closer look and found that does not mean charter schools are more integrated.

Race, policing and education during a summer on edge

This summer sadly provided no shortage of violence and heartache over issues that sometimes feel like they’re tearing America apart at the seams. We sought to bring some local perspective (and wisdom) to the debate by talking to an ambitious Manual High School student who took up a bullhorn at Denver street protests and to Aurora Public Schools Superintendent Rico Munn.

A middle school’s last-ditch effort to save itself 

An Adams County middle school running out of time to improve has placed its bet on more challenging, more personal teaching — and zero test preparation. Watch Chalkbeat later this week for a report on whether these efforts paid off in the form of improved state test scores. (Hopefully … the data are set to be made public Thursday).

Guess which Colorado school district had a high proportion of teachers designated to lose tenure …

Compared with other large Colorado school districts, Denver Public Schools had a higher proportion of teachers set to lose tenure under a sweeping educator effectiveness law passed six years ago. We surveyed big districts about one of the consequences of Senate Bill 191.

Too darn hot to teach and learn 

As part of its big bond request of voters this fall, Denver Public Schools wants to try to cool off some of its hottest schools. We took a look at where the mercury soars the highest and found that in 12 of the 18 hottest buildings — some of which house more than one school — the number of students who qualify for free or reduced price lunch exceeds the district average.

But the University Club has a lovely lunch menu (and squash courts, too)…  

What if the State Board of Education held a not-so-public meeting with the education commissioner at a private club downtown to prioritize goals, but didn’t get much of anything accomplished? That happened.

What we know — and don’t know — about Colorado remediation rates

Colorado’s college remediation rates inched upward after years of steady decline, a disheartening development. On top of that, we’re not getting the full picture, either, because of incomplete school-level numbers and non-existent district-level data.

Diet Coke: Coming soon to a high school vending machine near you? 

Despite opposition from advocacy groups, Colorado appears headed toward lifting a seven-year ban on diet soda in high schools. The rule change would clear the way for diet soda to be sold in high school vending machines and school stores, though districts could decide not to stock the drinks. We covered the issue before and after the State Board of Education’s initial vote.