First Person

Randomness is Not a Fluke

“I think there’s nothing wrong with anything.”  So spoke Chancellor Joel Klein at yesterday’s release of the 2009 elementary and middle school progress reports.  As Anna Phillips reported, 84% of the schools received a letter grade of A, and an additional 13% received a B.  Only two schools out of 1,058 received an F, and just five more were awarded a D.

The letter grades were driven by the remarkable/suspicious gains in 2009 on the state’s ELA and math tests.  Schools weren’t actually compared to one another on their performance this year to derive the letter grades.  Rather, they were compared to last year’s peer and citywide benchmarks.  To use a football metaphor, because test scores rose across the board, virtually all schools moved up the field, but the goalposts didn’t move.  I wasn’t sure that the progress report letter grades could actually be less useful this year than last, but Chancellor Klein’s administration has achieved that dubious feat.  When 84% of the schools receive an A—the top grade, which everyone understands to signify excellence—what useful information about the school’s relative performance is being conveyed to parents, students, educators, and others with a stake in our schools?  Not much, in my view.

Last year, my blogging partner Jennifer Jennings (who, for those keeping score at home, is now Dr. J) and I were sharply critical of the 2008 school progress reports.  Writing on Jennifer’s eduwonkette site, we demonstrated that student achievement growth over the past year—which makes up 60% of the overall progress report letter grade—was highly unreliable.  Schools that demonstrated high gains in student achievement from 2006 to 2007 were no more likely to show gains from 2007 to 2008 than schools that showed low gains in 2006 to 2007.  We concluded that the measure of student progress making up 60% of the overall progress report grade was picking up chance fluctuations from year to year.  And if 60% of the score is random, there’s not much genuine information about school performance in the progress report grade.

It wasn’t a fluke.

I’ve now replicated our analysis using the 2009 progress report grades.  Many measures of school performance—its environment, based on parent and teacher surveys, its attendance, even the percentage of students achieving Level 3 or Level 4 on the state ELA and math tests—are fairly stable from year to year.  But once again, the student progress measure is wholly unpredictable.  The schools that showed big gains in student achievement from 2007 to 2008 were no more likely to show big gains from 2008 to 2009 than those schools that had small gains from 2007 to 2008. 

Here are some simple graphical representations, using the roughly 600 elementary schools in New York City that received progress report grades in 2008 and 2009.  The first graph plots a school’s academic expectations in 2009, based on surveys, with that school’s academic expectations in 2008.  There is a strong positive correlation of .75, which indicates a great deal of stability from 2008 to 2009 in the relative position of a school compared to other schools.

acad-expec

The second graph plots a school’s attendance rate in 2009 with that school’s attendance rate in 2008.  Attendance is highly stable from year to year:  the correlation between 2008 attendance and 2009 attendance is .95.  There’s almost as much consistency in the percentage of students scoring at levels 3 or 4 on the state ELA test over these two years.  The correlation between the 2008 percentage and the 2009 percentage is .94, which is a very strong association indicating that schools that are relatively high in 2008 are likely to be relatively high in 2009.    

attendance
levels-3-4

As you look at the three graphs above, you can see a clear pattern in the data:  elementary schools that are relatively low compared to other schools in 2008 are also relatively low compared to their peers in 2009.  Similarly, elementary schools that are relatively high compared to other schools in 2008 are likely to be relatively high in comparison to other schools in 2009.  Now, look at the graph below, which plots each school’s student progress from 2008 to 2009 by that school’s progress from 2007 to 2008.  Do you see a pattern?

ela-growth

 If you do, there’s probably a psychologist somewhere salivating at the idea of administering a Rohrschach inkblot test to you.  The correlation between student progress from 2007 to 2008 and student progress from 2008 to 2009 is -.02, which is indistinguishable from zero.  This means that there really is no pattern to the results, and certainly not a pattern that demonstrates consistency or stability from one year to the next.

It’s no surprise to see this, since it follows from what we know about the instability of test score gains from one year to the next in schools across the country.  As was true last year, there’s a bit more consistency in progress in math, but the state ELA and math tests on which these scores are based are coming under increasing scrutiny for their predictability, easiness, and poor content coverage.  The New York City Department of Education has hitched its accountability wagon to a runaway horse.

When it comes to annual test score gains, randomness is not a fluke.

First Person

I’m a teacher in Memphis, and I know ‘grading floors’ aren’t a cheat — they’re a key motivator

PHOTO: Creative Commons / Shelly

Growing up, my father used to tell me not to come to him with a problem unless I had a solution.

That meant I learned quickly what kinds of solutions wouldn’t go over well — like ones involving my father and his money. His policy also meant that I had to weigh pros and cons, thinking about what I was able to do, what I wasn’t, and whom I needed help from in order to make things happen.

I sometimes wish decision-makers in Memphis had a father like mine. Because more often than not, it seems we are talking about the problems void of a solution or even possible solutions to vet.

Right now, the issue in Memphis and Shelby County Schools is the “grading floor,” or the policy of setting a lowest possible grade a teacher can assign a student. They have been temporarily banned after a controversy over high-school grade changing.

Grading floors aren’t new to teachers in Memphis, or to me, a fifth-grade teacher. I have taught and still teach students who are at least two grade levels behind. This was true when I taught fourth grade and when I taught sixth grade. Honestly, as the grade level increased, so did the gaps I saw.

More often than not, these students have been failed by a school, teacher, leader or system that did not adequately prepare them for the next grade. Meanwhile, in my classroom, I have a responsibility to teach grade-level material — adjusting it for individual students — and to grade their work accordingly.

That’s where “grading floors” come in. Without a grading floor, all of my current students would have grades below a 65 percent.

Can you imagine seeing the face of a fifth-grade boy who tried his hardest on your test, who answered all the questions you gave orally, who made connections to the text through auditory comprehension, only to receive a 0 on his paper?

I don’t have to imagine – I see similar reactions multiple times a day. Whether it’s a 65 percent or a 14 percent, it’s still an F, which signals to them “failure.” The difference between the two was summed up by Superintendent Hopson, who stated, “With a zero, it’s impossible to pass a course. It creates kids who don’t have hope, disciplinary issues; that creates a really bad scenario.”

I know that as years go by and a student’s proficiency gap increases, confidence decreases, too. With a lowered confidence comes a lower level of self-efficacy — the belief that they can do what they need to do to succeed. This, to me, is the argument for the grading floor.

In completing research for my master’s degree, I studied the correlation between reading comprehension scores and the use of a motivational curriculum. There was, as might have guessed, an increase in reading scores for students who received this additional curriculum.

So every day, I speak life into my students, who see Fs far too often in their daily lives. It is not my job as their teacher to eradicate their confidence, stifle their effort, and diminish their confidence by giving them “true” Fs.

“This is not an indication of your hard work, son. Yet, the reality is, we have to work harder,” I tell students. “We have to grind in order to make up what we’ve missed and I’m the best coach you have this year.”

In education, there are no absolutes, so I don’t propose implementing grading floors across the board. But I do understand their potential — not to make students appear more skilled than they are, or to make schools appear to be better than they are, but to keep students motivated enough to stay on track, even when it’s difficult.

If it is implemented, a grade floor must be coupled with data and other reports that provide parents, teachers, and other stakeholders with information that accurately highlights where a student is, both within the district and nationally. Parents shouldn’t see their child’s progress through rose-colored glasses, or be slapped by reality when options for their child are limited during and after high school.

But without hope, effort and attainment are impossible. If we can’t give hope to our kids, what are we here for?

I don’t have all the answers, but in the spirit of my father, don’t come with a problem unless you have a solution.

Marlena Little is a fifth-grade teacher in Memphis. A version of this piece first appeared on Memphis K-12, a blog for parents and students.

call out

Our readers had a lot to say in 2017. Make your voice heard in 2018.

PHOTO: Chris Hill/Whitney Achievement School
Teacher Carl Schneider walks children home in 2015 as part of the after-school walking program at Whitney Achievement Elementary School in Memphis. This photograph went viral and inspired a First Person reflection from Schneider in 2017.

Last year, some of our most popular pieces came from readers who told their stories in a series that we call First Person.

For instance, Carl Schneider wrote about the 2015 viral photograph that showed him walking his students home from school in a low-income neighborhood of Memphis. His perspective on what got lost in the shuffle continues to draw thousands of readers.

First Person is also a platform to influence policy. Recent high school graduate Anisah Karim described the pressure she felt to apply to 100 colleges in the quest for millions of dollars in scholarships. Because of her piece, the school board in Memphis is reviewing the so-called “million-dollar scholar” culture at some high schools.

Do you have a story to tell or a point to make? In 2018, we want to give an even greater voice to students, parents, teachers, administrators, advocates and others who are trying to improve public education in Tennessee. We’re looking for essays of 500 to 750 words grounded in personal experience.

Whether your piece is finished or you just have an idea to discuss, drop a line to Community Editor Caroline Bauman at [email protected]

But first, check out these top First Person pieces from Tennesseans in 2017:

My high school told me to apply to 100 colleges — and I almost lost myself in the process

“A counselor never tried to determine what the absolute best school for me would be. I wasted a lot of time, money and resources trying to figure that out. And I almost lost myself in the process.” —Anisah Karim     

Why I’m not anxious about where my kids go to school — but do worry about the segregation that surrounds us

“In fact, it will be a good thing for my boys to learn alongside children who are different from them in many ways — that is one advantage they will have that I did not, attending parochial schools in a lily-white suburb.” —Mary Jo Cramb

I covered Tennessee’s ed beat for Chalkbeat. Here’s what I learned.

“Apathy is often cited as a major problem facing education. That’s not the case in Tennessee.” —Grace Tatter

I went viral for walking my students home from school in Memphis. Here’s what got lost in the shuffle.

“When #blacklivesmatter is a controversial statement; when our black male students have a one in three chance of facing jail time; when kids in Memphis raised in the bottom fifth of the socioeconomic bracket have a 2.6 percent chance of climbing to the top fifth — our walking students home does not fix that, either.” —Carl Schneider

I think traditional public schools are the backbone of democracy. My child attends a charter school. Let’s talk.

“It was a complicated choice to make. The dialogue around school choice in Nashville, though, doesn’t often include much nuance — or many voices of parents like me.” —Aidan Hoyal

I grew up near Charlottesville and got a misleading education about Civil War history. Students deserve better.

“In my classroom discussions, the impetus for the Civil War was resigned to a debate over the balance of power between federal and state governments. Slavery was taught as a footnote to the cause of the war.” —Laura Faith Kebede