First Person

ELA Scores Are Up–But by How Much?

Yesterday, New York released the results of the 2009 English Language Arts (ELA) assessment administered in January to students in grades 3 through 8 across the state.  Overall, there was a sizeable increase from 2008 to 2009 in the percentage of students who achieved the standard of proficiency in ELA, from 68% of all students in grades 3 through 8 in 2008 to 77% in 2009.  In contrast, the scale scores which underlie the proficiency standards did not increase as much, rising an average of four points across the six grades in 2009, just one point more than the increase observed in 2008.

In New York City, the results were even more striking, with the percentage of students in grade 3 through 8 who reached the standard of proficiency in ELA rising from 58% in 2008 to 69% in 2009, an increase of 11 percentage points.  The increase in sixth grade is astonishingly large;  whereas 53% of New York City sixth-graders attained the proficiency standard in 2008, 73% did so in 2009, a whopping increase of 20 percentage points.  Taken at face value, figures such as these suggest that students’ English Language Arts achievement increased dramatically from 2008 to 2009.  Some observers, including me, worry that the increases exceed the gains in student learning that could reasonably be expected from changes in instruction occurring over the course of a single year.  The changes could reflect the kind of score inflation that testing expert Dan Koretz wrote about in his recent book Measuring Up. 

There are multiple ways to look at the distribution of ELA performance in 2008 and 2009, and other approaches yield a different estimate of the magnitude of the gains.  The percentage of students meeting the proficiency standard represents a binary threshold—a student is either above the bar or below it.  If you believe that the proficiency standard means something beyond the context of a particular test, then increasing the number and percentage of students who achieve that standard is an important accomplishment.  But the percentage of students who meet the standard tells you very little about the performance of low-achieving students who are well below the proficiency bar, and very little about the performance of high-achieving students who are well above the proficiency bar.  The average scale score takes into account the performance of all students who take the state ELA test, and changes in average scale scores over time may be as meaningful, and perhaps more so, than changes in proficiency rates.

New York City has an unfortunate habit of referring to changes over time on the state assessments as increases or decreases in “points,” where the points refer to percentage point differences in proficiency rates from year to year.  This is potentially misleading because percentage points are an arbitrary metric when applied to a threshold measure such as the percentage of students who are proficient.  Still, comparisons over time that are made in terms of percentage points aren’t going away.  The challenge is figuring out if they are large or small.

Another way of representing changes over time is to look at the extent to which the distribution of scale scores in a year has shifted from the distribution of scores in the prior year.  For example, we can look at the mean scale score for the ELA assessment in a given grade in 2009 and see how much larger or smaller it is than the mean scale score for that grade in 2008.  In New York City, the average scale scores increased from four to 10 points between 2008 and 2009.  Here again, the issue is whether this is a little or a lot. To gauge this, I’m representing the scale score increase in relation to the 2008 score distribution.  If the 2008 average scale score is at the 50th percentile of the 2008 distribution, and assuming that the 2008 distribution follows the familiar bell-shaped normal curve, how many percentiles higher is the average scale score in 2009?** 

The figure below shows both the percentage point increase between 2008 and 2009 in the percent of students who achieved the ELA proficiency standard in grades 3 through 8, as well as the percentile increase for the average score from 2008 to 2009.  In five of the six grades, the percentage point increase in the percent of students who achieved the proficiency standard exceeded the percentile increase from 2008 to 2009, and often substantially so.  Still, the percentile increases are nothing to sneeze at.


How might we explain why the percentage point increase in the percent of students who achieve the proficiency standard exceeds the percentile increase in the average scale score?  One possibility is that the gains that were observed from 2008 to 2009 were concentrated in the range of scores near the proficiency threshold.  Boosting the scores of students near the threshold just a bit could push them over the bar without dramatically increasing the overall distribution of achievement.  We can’t be sure that this is what’s happening, but the figure below gives a graphic representation of the process.  The phenomenon at issue is known as focusing on the “bubble kids,” those students whose performance is just below the threshold.  In the figure, moving the three “bubble kids” from just below the proficiency bar to just above it increases the percentage of students who are proficient from 43% to 67%, a 24 percentage point increase. My colleague Jennifer Jennings was influential in bringing to national attention the incentives that threshold-based accountability systems such as No Child Left Behind create for focusing on the bubble kids to the exclusion of other students.


I’d love to believe that the increases in performance observed on the ELA assessment in New York City and across the state represent real and enduring changes in what students know.  Seeing gains of similar magnitude on the 2009 National Assessment of Educational Progress would go a long way in convincing me.

**The percentile increases may be overly generous, because they are based on the amount of spread in the 2008 score distribution in New York City, which in some grades was substantially smaller than was observed in those grades in the years 2002 through 2007.  If the standard deviation used to calculate these percentile gains were larger, the percentile gains would appear smaller.

First Person

I’m a Bronx teacher, and I see up close what we all lose when undocumented students live with uncertainty

The author at her school.

It was our high school’s first graduation ceremony. Students were laughing as they lined up in front of the auditorium, their families cheering them on as they entered. We were there to celebrate their accomplishments and their futures.

Next to each student’s name on the back of those 2013 graduation programs was the college the student planned to attend in the fall. Two names, however, had noticeable blanks next to them.

But I was especially proud of these two students, whom I’ll call Sofia and Isabella. These young women started high school as English learners and were diagnosed with learning disabilities. Despite these obstacles, I have never seen two students work so hard.

By the time they graduated, they had two of the highest grade point averages in their class. It would have made sense for them to be college-bound. But neither would go to college. Because of their undocumented status, they did not qualify for financial aid, and, without aid, they could not afford it.

During this year’s State of the Union, I listened to President Trump’s nativist rhetoric and I thought of my students and the thousands of others in New York City who are undocumented. President Trump falsely portrayed them as gang members and killers. The truth is, they came to this country before they even understood politics and borders. They grew up in the U.S. They worked hard in school. In this case, they graduated with honors. They want to be doctors and teachers. Why won’t we let them?

Instead, as Trump works to repeal President Obama’s broader efforts to enfranchise these young people, their futures are plagued by uncertainty and fear. A Supreme Court move just last week means that young people enrolled in the Deferred Action for Childhood Arrivals program remain protected but in limbo.

While Trump and the Congress continue to struggle to find compromise on immigration, we have a unique opportunity here in New York State to help Dreamers. Recently, the Governor Cuomo proposed and the state Assembly passed New York’s DREAM Act, which would allow Sofia, Isabella, and their undocumented peers to access financial aid and pursue higher education on equal footing with their documented peers. Republicans in the New York State Senate, however, have refused to take up this bill, arguing that New York state has to prioritize the needs of American-born middle-class families.

This argument baffles me. In high school, Sofia worked hard to excel in math and science in order to become a radiologist. Isabella was so passionate about becoming a special education teacher that she spent her free periods volunteering with students with severe disabilities at the school co-located in our building.

These young people are Americans. True, they may not have been born here, but they have grown up here and seek to build their futures here. They are integral members of our communities.

By not passing the DREAM Act, it feels like lawmakers have decided that some of the young people that graduate from my school do not deserve the opportunity to achieve their dreams. I applaud the governor’s leadership, in partnership with the New York Assembly, to support Dreamers like Sofia and Isabella and I urge Senate Republicans to reconsider their opposition to the bill.

Today, Sofia and Isabella have been forced to find low-wage jobs, and our community and our state are the poorer for it.

Ilona Nanay is a 10th grade global history teacher and wellness coordinator at Mott Hall V in the Bronx. She is also a member of Educators for Excellence – New York.

First Person

I was an attorney representing school districts in contract talks. Here’s why I hope the Supreme Court doesn’t weaken teachers unions.

PHOTO: Creative Commons / supermac1961

Many so-called education reformers argue that collective bargaining — and unions — are obstacles to real change in education. It’s common to hear assertions about how “restrictive” contracts and “recalcitrant” unions put adult interests over children’s.

The underlying message: if union power were minimized and collective bargaining rights weakened or eliminated, school leaders would be able to enact sweeping changes that could disrupt public education’s status quo.

Those that subscribe to this view are eagerly awaiting the Supreme Court’s decision in the case of Janus v. American Federation of State, County, and Municipal Employees. At issue is the constitutionality of “agency” or “fair share” fees — employee payroll deductions that go to local unions, meant to cover the costs of negotiating and implementing a bargaining agreement.

In states that permit agency fees (there are about 20), a teacher may decline to be part of a union but must still pay those fees. If the Supreme Court rules that those agency fees are unconstitutional, and many teachers do not voluntarily pay, local unions will be deprived of resources needed to negotiate and enforce bargaining agreements.

Based on my experience as an attorney representing school districts in bargaining and contract issues, I have this to say to those hoping the Court will strike down these fees: be careful what you wish for.

Eliminating fair share fees (and trying to weaken unions) represents a misguided assumption about bargaining — that the process weakens school quality. To the contrary, strong relationships with unions, built through negotiations, can help create the conditions for student and school success. Indeed, in my experience, the best superintendents and school boards seized bargaining as an opportunity to advance their agenda, and engaged unions as partners whenever possible.

Why, and how, can this work? For one, the process of negotiations provides a forum for school leaders and teachers to hear one another’s concerns and goals. In my experience, this is most effective in districts that adopt “interest-based bargaining,” which encourages problem-solving as starting point for discussions as opposed to viewing bargaining as a zero-sum game.

Interest-based bargaining begins with both sides listing their major concerns and brainstorming solutions. The touchstone for a solution to be adopted in a bargaining agreement: Is the proposal in the best interests of children? This important question, if embedded in the process, forces both sides to carefully consider their shared mission.

For example, some districts I worked with paid teachers less than comparable neighboring districts did. It would have been unreasonable for unions to insist that their pay be increased enough to even that difference out, because that would mean reducing investments in other items of importance to children, like technology or infrastructure. At the same time, it would have been untenable for management to play “hard ball” and deny the problem, because to do so would likely lead to a disgruntled workforce.

Instead, both sides were forced to “own” the issue and collaboratively craft plausible solutions. That made unions more agreeable to proposals that demonstrated some commitment by the district to addressing the issue of pay, and districts open to other things that they could provide without breaking the budget (like more early release days for professional development).

To be sure, many school administrators could get frustrated with the process of bargaining or having to consult the negotiated agreement when they want to make a change. Some districts would very much like to adopt an extended school day, for example, but they know that they must first consult and negotiate such an idea with the union.

Yet, in districts where school administrators had built a reservoir of goodwill through collective bargaining, disagreement does not come at the cost of operating schools efficiently. Both sides come to recognize that while they inevitably will disagree on some things, they can also seek agreement — and often do on high-stakes matters, like teacher evaluations.

How does this relate to the Supreme Court’s pending decision? Without fees from some teachers, unions may lack the resources to ensure that contract negotiations and enforcement are robust and done well. This could create a vicious cycle: teachers who voluntarily pay fees for bargaining in a post-Janus world, assuming the court rules against the unions, will view such payments as not delivering any return on investment. In turn, they will stop contributing voluntarily, further degrading the quality of the union’s services.

Even more troubling, if fair share fees are prohibited, resentment and internal strife will arise between those who continue to pay the fees and those who refuse. This would undercut a primary benefit of bargaining — labor peace and a sense of shared purpose.

Speaking as a parent, this raises a serious concern: who wants to send their child to a school where there is an undercurrent of bitterness between teachers and administrators that will certainly carry over into the classroom?

It is easy to see the appeal of those opposing agency fees. No one wants to see more money going out of their paycheck. The union-as-bogeyman mentality is pervasive. Moreover, in my experience, some teachers (especially the newer ones) do not recognize the hidden benefits to bargaining contracts.

But, obvious or not, agency fees help promote a stable workplace that allows teachers to concentrate on their primary responsibility: their students. Removing the key ingredient threatens this balance.

Mark Paige is a former school teacher and school law attorney who represented school districts in New England. He is currently an associate professor of public policy at the University of Massachusetts – Dartmouth.