grading on a curve

Teachers give new Regents exam scoring system mixed reviews

The brand-new library at Evander Childs opened so teachers from other schools could grade Regents exams there.

Last year, the Evander Childs Campus got a new library, replete with rows of new computers and a mural depicting scholarly pursuits.

The library opened its doors for the first time last month — but not to students. Instead, it housed teachers from other high school campuses, who convened there to try out a new model for grading students’ final exams.

Regents exams, which students must pass to graduate from high school, have been scored by the teachers who administered them since the Regents exam program began in the nineteenth century. But mounting concerns about cheating — spurred on by the finding that students hit the minimum passing score at a disproportionately high rate — have prompted the city and state to make changes to how the exams are graded.

The state’s test security overhaul calls for schools to stop grading their own Regents exams by June 2013. The changes are meant to reduce opportunities and incentives for teachers to inflate their students’ scores, which under state law could factor into teachers’ evaluations in the future. The shift would bring Regents exam grading in line with how most states score high-stakes exams and with New York State’s requirements about elementary and middle schools’ exams.

Buoyed by its own concerns about cheating and softer forms of score inflation, the city has sped that timeline up. In January, a handful of schools tested out a system to ensure that teachers do not grade their own students’ exams.

Department of Education officials expanded that system, known as “distributed scoring,” to more than 160 schools this spring.  Most of the schools deployed teachers to centralized locations such as Evander Childs, and teachers from 17 schools tested a system for grading exams online. In total, about 107,000 exams were graded under distributed scoring last month.

Teachers who participated in the pilot gave it mixed reviews. Some said the system made them better graders because they considered only the answers, not the students, when assigning scores. But others said the system of musical graders was complicated, time-consuming, and likely to lead to unfairly deflated scores. And a small number of missing tests highlight the potential cost of logistical mishaps.

Department of Education officials solicited feedback from teachers who piloted the new system and said they would use that information to improve it before the next round of exams in 2013. Shael Polakow-Suransky, the department’s chief academic officer, said the pilot included a wide range of schools from across different districts and networks to elicit as full a range of feedback as possible.

“It was just trying to get a right mix so that we could actually see where the challenges will be and where we need to make adjustments for next year,” he said.

Concerns about fairness

For Richard Mangone, a retired social studies teacher recruited to grade U.S. History exams at the Prospect Heights Campus, few changes are needed. He said the scoring process at his site was the most efficient he had seen in 30 years of grading exams.

He also said he found it easier to grade fairly. One finding that prompted the city’s February audit was that teachers issued a disproportionate number of 65s — the lowest passing score — on Regents exams, suggesting that they might be bumping up the scores of students on the verge of passing. That’s less likely to happen now that teachers are not grading their own students’ tests, Mangone said.

“It’s not that you’re less objective, but it’s easier,” Mangone said about distributed scoring. “You’re just looking at the response.”

Some said have argued the bulge of 65’s reflects not padded scores but concern for students most at risk of failing. At a panel last August, high school social studies teacher (and GothamSchools Community section contributor) Stephen Lazar said that when teachers are more invested in their success, they are more attentive while grading, preventing careless scoring errors from costing a student the score he needs to graduate.

Monica Mazzocchi, who teaches at New Utrecht High School, which was a scoring center, said she prefers grading her own exams for the same reasons.

“Because it’s not their students, will they care as much as we care?” she said about other graders.

Other teachers said the lack of context could be problematic for other reasons. Peter Lapré, a social studies teacher at Park East High School, said he teaches his students extensively about the Venetian salt trade, even though the subject is not covered in standard course materials. This year, his students’ exams were graded online by teachers at other schools — some of whom might not be familiar with that topic.

“I’m concerned my students who chose to write about that were graded unfairly because the teacher didn’t know that information,” he said.

A teacher from Harry S. Truman High School said she worried that other graders wouldn’t be aware that students from her school take U.S. history in ninth grade rather than in their junior year, as students in most schools do, and would grade them according to the standards they would apply to high school juniors.

Fears of score deflation

The Truman teacher, who scored exams at DeWitt Clinton High School and asked not to be named because she feared repercussions, said she found that her concerns were warranted: Her school’s test scores in the subjects graded at the central location dropped significantly, even though they rose in subjects graded in the old model. And Lapré said his scores were stable from last year, even though his school had doubled the time devoted to global studies instruction in an effort to boost scores.

Both teachers said they thought the new system placed more pressure on teachers to grade harshly by exposing them to oversight from their colleagues and supervisors. The Truman teacher said that because no one at her site wanted to be seen as too lax, teachers debating between two scores usually tended to round down instead of up.

Plus, each grader was assigned a three-digit identification number and assigned to write them next to every response he or she scored.

“You’re well aware you’re being watched,” Lapré said.

Each exam was also marked with the student’s name and school. Arthur Goldstein, a social studies teacher at Francis Lewis High School, which did not participate in the pilot, said he was concerned that information could bias graders against students.

“I wonder if a bunch of papers go to a closing school [to be graded], if they won’t look at it and make my kids pay for it because we’re a good school,” he said.

Another teacher whose school did not participate in the pilot said he worried that bias could cut the other way, disadvantaging students whose names or schools suggested they were likely to be black or Hispanic because teachers would expect them to perform less well.

The city’s progress report system for evaluating schools judges high schools in large part by their Regents exam pass rates, and rates that fall from one year to the next would result in a lower grade. The system also weighs each school’s performance against that of other schools with similar students. When schools’ annual letter grades are announced this fall, some schools that used distributed scoring will have been compared to schools whose exams were scored under the old system.

Department officials said the distributed grading model actually shields students from unfairness. Before teachers even began grading tests at the centralized sites, they completed an exercise to make sure they shared an understanding about what makes an essay worth one score rather than another. First, each teacher graded the same essay, and then members of each grading discussed their rationales before comparing their assessments to the state’s guidelines.

The actual scoring happened in committees of four or six, with two teachers grading every essay. Discrepancies of more than one point between the teachers’ scores would trigger another reading by a third teacher, according to a department official who worked on the new system. That rarely happened, teachers who participated in the pilot said.

A large-scale logistical undertaking

Test scores weren’t the only things that moved as a result of the pilot: The physical tests also had to be transported around the city, posing a logistical challenge. In one extreme hiccup, 17 exams taken at Franklin D. Roosevelt High School in Brooklyn that were supposed to be taken to New Utrecht for grading were lost.

The Department of Education’s Office of Special Investigation is looking into what happened and whether distributed scoring played a role, according to Marge Feinberg, a department spokeswoman. She said the exams, which were taken by students in FDR’s evening school for students at risk of dropping out, are the only ones missing from June’s Regents period.

“We are working with school staff to find the exams,” Feinberg said.

Other logistical issues had graders concerned about distributed scoring’s efficiency. The teacher from Truman said she and her colleagues spent long stretches in DeWitt Clinton’s library, doing nothing while they waited for exams to be collected or distributed.

A Brooklyn teacher who commuted with 24 colleagues to score at New Utrecht said he said he found it inefficient to commute each day to a school other than his own. And he worried that the presence of 100 extra people in the building while New Utrecht’s students were taking final exams was disruptive.

But the teacher, who asked not to be named because he feared repercussions, said he saw a value in handing off his students’ exams for others to grade.

“What I would recommend personally is, give me Utrecht’s, I’ll give them my papers, and we can stay in our own buildings,” the teacher said.

Behind the brawl

Three things to know about the Tennessee school behind this week’s graduation brawl

PHOTO: Arlington Community Schools
Arlington High School is a 2,000-plus-student school in suburban Shelby County in southwest Tennessee.

Arlington High School is considered the crown jewel of a 3-year-old district in suburban Shelby County, even as its school community deals with the unwelcome attention of several viral videos showing a fight that broke out among adults attending its graduation ceremony.

The brawl, which reportedly began with a dispute over saved seats, detracted from Tuesday’s pomp and circumstance and the more than $30 million in scholarships earned by the school’s Class of 2017. No students were involved.

“It was unfortunate that a couple of adults in the audience exhibited the behavior they did prior to the ceremony beginning and thus has caused a distraction from the celebration of our students’ accomplishments,” Arlington Community Schools Superintendent Tammy Mason said in a statement.

Here are three things to know about the 13-year-old school in northwest Shelby County.

With more than 2,000 students, Arlington is one of the largest high schools in Shelby County and is part of a relatively new district.

It’s the pride of a suburban municipality that is one of six that seceded from Shelby County Schools in 2014 following the merger of the city and county districts the year before. (School district secessions are a national trend, usually of predominantly white communities leaving predominantly black urban school systems.) More than 70 percent of Arlington’s students are white, and 6 percent are considered economically disadvantaged — in stark contrast to the Memphis district where less than 8 percent are white, and almost 60 percent are considered economically disadvantaged.

The school’s graduation rate is high … and climbing.

Last year, after adding interventions for struggling students, the school’s graduation rate jumped a full point to more than 96 percent. Its students taking the ACT college entrance exam scored an average composite of 22.5 out of a possible 36, higher than the state average of 19.9. But only a fifth scored proficient or advanced in math and a third in English language arts during 2015-16, the last school year for which scores are available and a transition year for Tennessee under a new test.

PHOTO: Caroline Bauman
Education Commissioner Candice McQueen visits with students at Arlington High School during a 2016 tour.

The school was in the news last August when Tennessee Education Commissioner Candice McQueen visited its campus.

The commissioner spoke with students there to kick off her statewide listening tour that’s focused on ways to get students ready for college and career. McQueen highlighted the school’s extracurricular activities and students’  opportunities to intern for or shadow local professionals. She also complimented Arlington for having an engaged education community. 

poster campaign

How one Memphis student is elevating the conversation about school discipline

PHOTO: Laura Faith Kebede
Posters created by junior Janiya Douglas have amplified student voices about the culture of White Station High School in Memphis.

Now in her third year of attending a premier public high school in Memphis, Janiya Douglas says she’s observed discipline being handed out unevenly to her classmates, depending on whether they are on the college preparatory track.

PHOTO: Laura Faith Kebede
From left: Janiya Douglas and Michal Mckay are student leaders in Bridge Builders CHANGE program.

“We’re heavily divided in an academic hierarchy,” said Janiya, a junior in the optional program for high-achieving students at White Station High School. “It’s obvious students are treated differently if they are in traditional classes.”

Janiya also has observed racial disparities in how students are disciplined, and the state’s data backs that up. White Station students who are black or Hispanic are suspended at significantly higher rates than students who are white.

Frustrated by what she’s seen, Janiya took her concerns last Friday to the hallways of White Station and hung 14 posters to declare that “our school doesn’t treat everybody equally.”

By Monday morning, the posters were gone — removed by school administrators because Janiya did not get prior approval — but not before other students shared images of some of the messages on social media.

Now, Janiya is seeing some fruits of her activism, spawned by her participation in Bridge Builders CHANGE, a student leadership program offered by a local nonprofit organization.

In the last week, she’s met with Principal David Mansfield, a school counselor and a district discipline specialist to discuss her concerns. She’s encouraged that someone is listening, and hopes wider conversations will follow.

The discussions also are bringing attention to an online petition by the education justice arm of Bridge Builders calling for suspension alternatives across schools in Memphis.

White Station often is cited as one of the jewels of Shelby County Schools, a district wrought with academic challenges. The East Memphis school is partially optional, meaning some students test into the college prep program from across the county.

But Janiya and some of her classmates say they also see an academically and racially segregated school where students zoned to the traditional program are looked down upon by teachers. Those students often get harsher punishments, they say, than their optional program counterparts for the same actions.

“Our school doesn’t treat everybody equally. A lot of groups aren’t treated equally in our school system,” junior Tyra Akoto said in a quote featured on one poster.

“If we get wrong with a teacher, they’ll probably write us up. But if a white student was to do it, they’ll just play it off or something like that,” said Kelsey Brown, another junior, also quoted in the poster campaign.

A district spokeswoman did not respond to questions about disciplinary issues raised by the posters, but offered a statement about their removal from the school’s walls.

White Station is known for “enabling student voice and allowing students to express their opinions in various ways,” the statement reads. “However, there are protocols in place that must be followed before placing signs, posters, or other messages on school property. Schools administrators will always work with students to ensure they feel their voices are heard.”

PHOTO: @edj.youth/Instagram
Members of the education justice arm of the Bridge Builders CHANGE program

To create the posters, Janiya interviewed about two dozen students and had been learning about about school discipline disparities as part of the Bridge Builders CHANGE program.

State discipline data does not differentiate academic subgroups in optional schools. But white students in Shelby County Schools are more likely to be in an optional school program and less likely to be suspended. And statewide in 2014-15, black students were more than five times as likely as white students to be suspended.

White Station reflects those same disparities. About 28 percent of black boys and 19 percent of black girls were suspended that same year — significantly higher than the school’s overall suspension rate of 14 percent. About 17 percent of Hispanic boys and 7 percent of Hispanic girls were suspended. By comparison, 9 percent of white boys and 2 percent of white girls were suspended.

Shelby County Schools has been working to overhaul its disciplinary practices to move from punitive practices to a “restorative justice” approach — a transition that is not as widespread as officials would like, according to Gina True, one of four specialists implementing a behavior system called Positive Behavior Interventions and Supports, or PBIS.

“The whole goal is to not get them suspended, because we want to educate them,” said True, who met this week with Janiya and several other students from Bridge Builders. “When students are cared for emotionally, they perform better academically. As counselors, that’s what we’ve been saying for years.”

Janiya acknowledges that she didn’t follow her school’s policy last week when hanging posters without permission at White Station. But she thinks her action has been a catalyst for hard conversations that need to happen. And she hopes the discussions will include more student input from her school — and across the district.

“Those most affected by the issues should always be a part of the solution,” she said.

Correction: April 10, 2017: A previous version of this story said Janiya put up 50 posters at her school. She designed 50 but actually posted only 14.