A slew of glitches in the city’s electronic grading for Regents exams have delayed scores for several subjects, just days before high schools are set to begin holding graduation ceremonies.
The problems represent at best a significant inconvenience and cost and at worst a threat to students’ scores and graduation status, according to educators with knowledge of the grading process.
This is the first June that all Regents exams taken at city high schools are being graded through “distributed scoring,” an arrangement devised to prevent teachers from scoring tests taken by students at their schools. Until last year, teachers graded their own students’ exams, but under pressure to show that test scores are not inflated, the state barred that practice. The city’s scoring system extends the state’s rules.
After a pilot last year, the Department of Education opted to have four of the most-taken tests — Living Environment, Global Studies, U.S. History, and English — scored electronically. McGraw-Hill, the vendor administering the process, collects the exams at schools, transports them to a scanning site in Connecticut, and then distributes answers one by one to teachers stationed at computers in city grading centers.
The company is getting $3.5 million this year from the city to administer the distributed scoring program, part of a $9.6 million, three-year contract to manage the logistical acrobatics that the new arrangement requires.
The process resulted in quicker and scoring during the pilot, according to Adina Lopatin, the department’s deputy chief academic officer. She said teachers moved more quickly through exam responses because they did not have to shuffle through papers.
But the time savings have been more than negated by serious glitches as the city has scaled up electronic scoring to include all high schools this month. At some schools, exams were not picked up until days after they were taken, teachers said. Teachers have reported to scoring sites daily, only to be sent back to their schools after being told that not enough items had been scanned for them to grade.
And even when there are answers to score, bandwidth issues have prevented teachers from grading them quickly in some schools, and in others, McGraw-Hill’s efforts to redact identifying information about students left answers partially obscured. One teacher said his site had graded just 20 exams on Monday and another 50 today because of the problems.
The cumulative result is that three exams required for graduation will not be graded by the department’s expected deadline. Niket Mull, who directs the Office of Assessment, sent an email to principals on Monday explaining that the Living Environment exam would be scored not by Wednesday but by Friday. Instead of being scored by Thursday, the history exams will not be complete until the end of the week or even Monday.
Schools begin holding graduation ceremonies on Thursday, although most are scheduled for next week. Schools are also in the process of determining who needs to attend summer school and what courses students should take next fall, decisions that can’t be made until scores are in.
One principal who estimated that at least 50 students at his school need scores to graduate that seem unlikely to arrive in time for the school’s commencement. He said he would be asking their parents to sign a form indicating that they understand that graduation is conditional on the test scores.
“I’ve never had to do that before,” the principal said. “I’m pretty strict about allowing kids to walk if they haven’t met the requirements. … But this is different.”
That’s the process that the department will be advising principals to use, softening a normally hard and fast rule that students cannot walk at graduation if they have not met all of the requirements, according to Erin Hughes, a spokeswoman.
Hughes said McGraw-Hill would pay for teachers’ overtime pay if scoring has to happen over the weekend, something she said she was not sure would happen.
When the department put out a call for teachers to grade the global history exam in January, only about 60 percent as many signed up as were needed, and the city was forced to drop the requirement that schools not grade their own tests. Many, many more exams are taken in June.
A teacher who was sent home early from a Brooklyn grading site on Monday after exhausting the answers in the online system said he was told that weekend scoring would be inevitable. Only about a quarter of the city’s 60,000 global studies exams have been graded, he said administrators at the site told graders today.
He said he also worried that the electronic scoring system would adversely affect students’ scores. Many of the essays he saw had the first several lines blocked out by McGraw-Hill’s effort to obscure details about students, and he said he marked all of the essays as unreadable.
Another issue, teachers said, is that graders might have marked essays as unwritten if students began them in the wrong place in the test booklet, or as not responding to the question if students wrote the wrong essay in each space.
“Back in the paper days you could flip around the booklet to see what else the student wrote, and figure out if the first essay was in fact the DBQ, and the essay in “Part IIIB” page was the thematic,” said a teacher with eight years of experience grading Regents exams, referring to the two essay types on history exams.
And a third teacher said bandwidth issues meant that several minutes elapsed at times between when graders asked for the next page to load and when it appeared on their screen. “This is asking a teacher with tired eyes from staring at a computer screen to remember that essay they were reading and pick up later,” the teacher said. “I can’t imagine this … having no negative effects on students.”
Overall, teachers said, the glitches raised serious questions about the city’s decision to outsource a grading process that had always been done in-house.
“We could have done this already if we’d had the exams in the school,” one teacher said. “It’s very unsettling and not good for the students.”