First Person

ELA Scores Are Up–But by How Much?

Yesterday, New York released the results of the 2009 English Language Arts (ELA) assessment administered in January to students in grades 3 through 8 across the state.  Overall, there was a sizeable increase from 2008 to 2009 in the percentage of students who achieved the standard of proficiency in ELA, from 68% of all students in grades 3 through 8 in 2008 to 77% in 2009.  In contrast, the scale scores which underlie the proficiency standards did not increase as much, rising an average of four points across the six grades in 2009, just one point more than the increase observed in 2008.

In New York City, the results were even more striking, with the percentage of students in grade 3 through 8 who reached the standard of proficiency in ELA rising from 58% in 2008 to 69% in 2009, an increase of 11 percentage points.  The increase in sixth grade is astonishingly large;  whereas 53% of New York City sixth-graders attained the proficiency standard in 2008, 73% did so in 2009, a whopping increase of 20 percentage points.  Taken at face value, figures such as these suggest that students’ English Language Arts achievement increased dramatically from 2008 to 2009.  Some observers, including me, worry that the increases exceed the gains in student learning that could reasonably be expected from changes in instruction occurring over the course of a single year.  The changes could reflect the kind of score inflation that testing expert Dan Koretz wrote about in his recent book Measuring Up. 

There are multiple ways to look at the distribution of ELA performance in 2008 and 2009, and other approaches yield a different estimate of the magnitude of the gains.  The percentage of students meeting the proficiency standard represents a binary threshold—a student is either above the bar or below it.  If you believe that the proficiency standard means something beyond the context of a particular test, then increasing the number and percentage of students who achieve that standard is an important accomplishment.  But the percentage of students who meet the standard tells you very little about the performance of low-achieving students who are well below the proficiency bar, and very little about the performance of high-achieving students who are well above the proficiency bar.  The average scale score takes into account the performance of all students who take the state ELA test, and changes in average scale scores over time may be as meaningful, and perhaps more so, than changes in proficiency rates.

New York City has an unfortunate habit of referring to changes over time on the state assessments as increases or decreases in “points,” where the points refer to percentage point differences in proficiency rates from year to year.  This is potentially misleading because percentage points are an arbitrary metric when applied to a threshold measure such as the percentage of students who are proficient.  Still, comparisons over time that are made in terms of percentage points aren’t going away.  The challenge is figuring out if they are large or small.

Another way of representing changes over time is to look at the extent to which the distribution of scale scores in a year has shifted from the distribution of scores in the prior year.  For example, we can look at the mean scale score for the ELA assessment in a given grade in 2009 and see how much larger or smaller it is than the mean scale score for that grade in 2008.  In New York City, the average scale scores increased from four to 10 points between 2008 and 2009.  Here again, the issue is whether this is a little or a lot. To gauge this, I’m representing the scale score increase in relation to the 2008 score distribution.  If the 2008 average scale score is at the 50th percentile of the 2008 distribution, and assuming that the 2008 distribution follows the familiar bell-shaped normal curve, how many percentiles higher is the average scale score in 2009?** 

The figure below shows both the percentage point increase between 2008 and 2009 in the percent of students who achieved the ELA proficiency standard in grades 3 through 8, as well as the percentile increase for the average score from 2008 to 2009.  In five of the six grades, the percentage point increase in the percent of students who achieved the proficiency standard exceeded the percentile increase from 2008 to 2009, and often substantially so.  Still, the percentile increases are nothing to sneeze at.

ela-nyc

How might we explain why the percentage point increase in the percent of students who achieve the proficiency standard exceeds the percentile increase in the average scale score?  One possibility is that the gains that were observed from 2008 to 2009 were concentrated in the range of scores near the proficiency threshold.  Boosting the scores of students near the threshold just a bit could push them over the bar without dramatically increasing the overall distribution of achievement.  We can’t be sure that this is what’s happening, but the figure below gives a graphic representation of the process.  The phenomenon at issue is known as focusing on the “bubble kids,” those students whose performance is just below the threshold.  In the figure, moving the three “bubble kids” from just below the proficiency bar to just above it increases the percentage of students who are proficient from 43% to 67%, a 24 percentage point increase. My colleague Jennifer Jennings was influential in bringing to national attention the incentives that threshold-based accountability systems such as No Child Left Behind create for focusing on the bubble kids to the exclusion of other students.

bubble-kids

I’d love to believe that the increases in performance observed on the ELA assessment in New York City and across the state represent real and enduring changes in what students know.  Seeing gains of similar magnitude on the 2009 National Assessment of Educational Progress would go a long way in convincing me.

**The percentile increases may be overly generous, because they are based on the amount of spread in the 2008 score distribution in New York City, which in some grades was substantially smaller than was observed in those grades in the years 2002 through 2007.  If the standard deviation used to calculate these percentile gains were larger, the percentile gains would appear smaller.

First Person

I’m a black male teen in Aurora, and I see how ‘achievement gap’ forms

The author, Ayden Clayton.

Have you ever heard of the achievement gap? Every column, blog or article that I’ve read on this topic has never come from a African-American, let alone an African-American male.

Here is a voice that should be heard: mine.

Recent research from Stanford showed that African-Americans come in behind other students on standardized tests and enrollment in honors to AP and college classes. This is very important because the gap is also prevalent at Rangeview High School in Aurora, where I am a senior.

There really is a problem. Look at the facts: 25.8 percent of African Americans are in poverty according to Census information published in 2013. The problem is how their lives at home are affecting classroom behavior or attention in class. This goes for all races, but the trend is that many of the students with families living in poverty drop out of high school.

“I believe the achievement gap is a multi-level problem in the education system,” English teacher Mr. Jordan Carter, who works at Rangeview and is a mixed minority, told me. “The hardest thing about it is telling people it is a significant problem. We can solve it by devoting time and resources to find the problem and we need to address kids from all backgrounds. Kids with better resources usually do better.”

I see other problems, too. As a student at Rangeview, I’ve been in numerous AP, honors and CCA classes (college courses) throughout my high school career. What I really have noticed were the underprivileged kids being treated differently, almost like the teachers thought of them as troublemakers without even knowing them.

I’ve had many teachers stereotype me about drugs, hip-hop, if I have a dad and more, and it made me pretty uncomfortable to the point where I didn’t want to go to the class. I feel that when issues such as these that occur in the classroom, it makes students of color not want to focus, and teachers could probably use better training on how to teach kids that do not look like them.

Those students would continuously sit in the back of classes, wouldn’t raise their hand, and wouldn’t ask questions. I used to be one of them. It’s not because the urge to not learn, but the discomfort of the setting in the classroom. When you get looked at and thought of like that, you don’t feel welcomed.

It is becoming evident that Rangeview is in need of a serious sit-down with some of our staff, such as the principal, teachers and all administrators. That way, students can see where their minds are and how they are trying to deal with the way they feel about fair conditions in the classroom.

The administrators should also talk to students – particularly minority students – about our wants and needs so we as students can have some input. For the students who are struggling, it would be great to have counselors talk to them and find a way that would help the students improve their academic careers, such as tutoring or staying after school.

I have faced the stereotype of being another dropout who is eventually going to jail, but I use that as inspiration every day. I know that all African-American males and females can make a change by letting our voice be heard.

Although I haven’t been through as much as other African-American students, I’ve been through enough to have my opinion matter. We — as minorities — can also take responsibility to change this problem by staying in school and voting into our government people who will fund impoverished areas.

As a community we need to fight stereotypes together. We either defeat stereotypes together or become the stereotypes ourselves.

Ayden Clayton is a senior at Rangeview High School. This piece first appeared in the Rangeview Raider Review.

First Person

Why the phrase ‘with fidelity’ is an affront to good teaching

PHOTO: Alan Petersime

“With fidelity” are some of the most damaging words in education.

Districts spend a ton of money paying people to pick out massively expensive, packaged curriculums, as if every one of a thousand classrooms needs the exact same things. Then officials say, over and over again, that they must be implemented “with fidelity.” What they mean is that teachers better not do anything that would serve their students’ specific needs.

When that curriculum does nothing to increase student achievement, it is not blamed. The district person who found it and purchased it is never blamed. Nope. They say, “Well, the teachers must not have been implementing it with fidelity.”

It keeps happening because admitting that schools are messy and students are human and teaching is both creative and artistic would also mean you have to trust teachers and let them have some power. Also, there are some really crappy teachers out there, and programs for everyone are often meant to push that worst-case-scenario line a little higher.

And if everyone’s doing just what they’re supposed to, we’ll get such good, clean numbers, and isn’t that worth a few thousand more dollars?

I was talking with a friend recently, a teacher at an urban school on the East Coast. He had been called to task by his principal for splitting his kids into groups to offer differentiated math instruction based on students’ needs. “But,” the principal said, “did the pacing guide say to differentiate? You need to trust the system.”

I understand the desire to find out if a curriculum “works.” But I don’t trust anyone who can say “trust the system” without vomiting. Not when the system is so much worse than anything teachers would put together.

Last year, my old district implemented Reading Plus, an online reading program that forces students to read at a pace determined by their scores. The trainers promised, literally promised us, that there wasn’t a single reading selection anywhere in the program that could be considered offensive to anyone. God knows I never learned anything from a book that made me feel uncomfortable!

Oh, and students were supposed to use this program — forced-paced reading of benign material followed by multiple-choice questions and more forced-pace reading — for 90 minutes a week. We heard a lot about fidelity when the program did almost nothing for students (and, I believe quite strongly, did far worse than encouraging independent reading of high-interest books for 90 minutes a week would have done).

At the end of that year, I was handed copies of next year’s great adventure in fidelity. I’m not in that district any longer, but the whole district was all switching over to SpringBoard, another curriculum, in language arts classes. On came the emails about implementing with fidelity and getting everyone on the same page. We were promised flexibility, you know, so long as we also stuck to the pacing guide of the workbook.

I gave it a look, I did, because only idiots turn down potential tools. But man, it seemed custom-built to keep thinking — especially any creative, critical thought from either students or teachers — to a bare minimum.

I just got an email from two students from last year. They said hi, told me they missed creative writing class, and said they hated SpringBoard, the “evil twin of Reading Plus.”

That district ran out of money and had to cut teachers (including me) at the end of the year. But if they hadn’t, I don’t think I would have lasted long if forced to teach from a pacing guide. I’m a good teacher. Good teachers love to be challenged and supported. They take feedback well, but man do we hate mandates for stuff we know isn’t best for the kids in our room.

Because, from inside a classroom full of dynamic, chaotic brilliance;

from a classroom where that kid just shared that thing that broke all of our hearts;

from a classroom where that other kid figured out that idea they’ve been working on for weeks;

from that classroom where that other kid, who doesn’t know enough of the language, hides how hard he works to keep up and still misses things;

and from that classroom where one kid isn’t sure if they trust you yet, and that other kid trusts you too much, too easily, because their bar had been set too low after years of teachers that didn’t care enough;

from inside that classroom, it’s impossible to trust that anyone else has a better idea than I do about what my students need to do for our next 50 minutes.

Tom Rademacher is a teacher living in Minneapolis who was named Minnesota’s Teacher of the Year in 2014. His book, “It Won’t Be Easy: An Exceedingly Honest (and Slightly Unprofessional) Love Letter to Teaching,” was published in April. He can be found on Twitter @mrtomrad and writes on misterrad.tumblr.com, where this post first appeared.