First Person

The Right To Know What?

Each fall, thousands of runners descend on the Big Apple to run the New York City marathon. They’ve trained hard all year, and give their all on the course. Long after the elite runners have finished, they stream across the finish line in clumps, exhausted at the end of their 26.2-mile journey. In the middle of the pack, as many as eight or 10 runners might cross the finish line in a single second, and nearly 400 in a single minute.

The difference between a time of 4:08:00 and 4:09:00, however, isn’t large enough to be important. It’s the difference between a rate of 9:28 per mile and 9:30 per mile. Given the vagaries of marathon running — the wind, the temperature, the features of the course — it would be unwise to conclude that the runner who crossed the finish line in 4:08:00 is a much better marathoner than the one who finished in 4:09:00.

But the runner with a time of 4:08:00 finished several hundred places ahead of the runner who finished in 4:09:00 — surely that counts for something! Not really, I’d say. We can quantify the difference, both in absolute terms and in relative position, but these differences are not large enough to be meaningful.

The same is true of the information in the Teacher Data Reports recently released in New York City. Small differences in the estimated effects of teachers on their students’ achievement can appear to be much larger, because most teachers are about equally successful with the assortment of students they teach in a given year, regardless of whether those students begin the year as low-achievers or high-achievers. A trivial difference can appear much larger than it actually is, because, like the marathoners, many teachers are “crossing the finish line” at about the same time.

Here’s an example drawn from the 2008-09 Teacher Data Reports. (I chose the example because it’s convenient and have no reason to believe it’s unusual.) In 2009, fifth-graders took New York State’s English Language Arts exam, which consisted of 24 multiple-choice test items and three constructed-response items, which were combined to create a raw score ranging from 0 to 31. The raw scores were then converted to scale scores, which were used to classify individual students into four levels of performance, with Level 3 representing grade-level proficiency. The average student in New York City got 25.5 raw score points out of 31, which in New York City’s scheme represented an average proficiency level of 3.29. (Sounds pretty good, right? Of course, this was before the state wised up that being proficient on the test didn’t mean a student was on track to graduate from high school ready for college.)

The logic of the city’s Teacher Data Reports is to estimate Teacher A’s contribution to his or her students’ test-scores by comparing how other students with the same measured characteristics would be expected to do on the state test, based on their prior achievement and individual and classroom characteristics, with how Teacher A’s students actually did on the test. If Teacher A’s students score at the same level as was predicted by a statistical model, Teacher A is claimed to not “add value” to her students. If Teacher B’s students perform better than expected, Teacher B is said to add value. (And poor Teacher C, whose students score lower than they are predicted to do, is subtracting value, I guess. Maybe we should call him Teacher F.) These “value-added” scores are then ranked, and a teacher is assigned a percentile value representing the percentage of other teachers teaching the same grade and subject who scored below he or she did.

An “average” teacher, according to this calculation, is one whose value-added score is 0. Of the 1,751 NYC teachers with three or more years of experience who received a value-added rating in fifth-grade English in 2008-09, 84 got a score that rounded to .00. Their percentile ratings—the number that’s getting all of the attention in the traditional and neo-tabloids—range from 53 to 58. A tiny shift of .01 in either direction yields an additional 152 teachers, and a percentile rating of 48 to 63. What seems to be a small range of value-added scores could be anywhere from the 48th to the 63th percentile, because the value-added scores in this range are clumped together.

But it’s hard to know whether a shift of .01 in either direction is large or small. How can we tell? Here’s an idea. Suppose that Ruiz had 20 students who took the fifth-grade English test in 2009, and they were at the city average of 25.5 out of 31 raw score points on the test. What if half of the students got one more question right on the test? Doesn’t seem like a big stretch, does it? Just like the variation in the conditions on marathon day, half of the students getting one more question correct on a given test on a given day doesn’t seem out of the realm of possibility.

If this were to happen, Ruiz’s value-added score would rise from 0 to .05. And the percentile range associated with a value-added score of .05 is 75 to 77. All of a sudden, an “average” teacher looks pretty good. And this isn’t due to the margin of error! It’s just because many teachers are about equally effective in promoting student achievement, according to the value-added model in use. A relatively small change in student performance shifts a teacher’s location in the value-added distribution by a surprisingly large amount.

To be sure, this example is based on one year of student test-score data, not multiple years. But that’s what New York State is proposing to rely on in its first year of the new Annual Professional Performance Review process, and it’s what other jurisdictions, such as Washington, D.C., use in their teacher-evaluation systems. And, as with the marathon, the clumping together of teachers is more of an issue in the middle of the distribution than among those in the lead or at the back of the pack. But that’s little consolation to the teachers whose percentile rankings will figure into annual evaluations that will determine whether they’re permitted to continue teaching.

Speaking at Coney Island Feb. 28, Mayor Bloomberg defiantly affirmed the public’s right to know the contents of teachers’ performance evaluations. “Parents have a right to know every bit of information that we can possibly collect about the teacher that’s in front of their kids,” he said.

That statement is utterly ridiculous. There’s no legitimate interest in information about teachers’ private lives if it has no bearing on their professional performance. But here’s something parents do have the right to know: just how fragile value-added measures based on the New York State testing system are. The New York State tests were never intended to be used to rate teachers’ contributions to student learning — and so it’s little wonder they do a pretty poor job of it.

This post also appears on Eye on Education, Aaron Pallas’s Hechinger Report blog.

First Person

Two fewer testing days in New York? Thank goodness. Here’s what else our students need

PHOTO: Christina Veiga

Every April, I feel the tension in my fifth-grade classroom rise. Students are concerned that all of their hard work throughout the year will boil down to six intense days of testing — three for math and three for English language arts.

Students know they need to be prepared to sit in a room for anywhere from 90 minutes to three hours with no opportunity to leave, barring an emergency. Many of them are sick to their stomachs, feeling more stress than a 10-year-old ever should, and yet they are expected to perform their best.

Meanwhile, teachers are frustrated that so many hours of valuable instruction have been replaced by testing, and that the results won’t be available until students are moving on to other classrooms.

This is what testing looks like in New York state. Or, at least it did. Last month, state officials voted to reduce testing from three days for each subject to two, to the elation of students, parents, and teachers across New York. It’s an example of our voices being heard — but there is still more to be done to make the testing process truly useful, and less stressful, for all of us.

As a fifth-grade teacher in the Bronx, I was thrilled by the news that testing time would be reduced. Though it doesn’t seem like much on paper, having two fewer days of gut-wrenching stress for students as young as eight means so much for their well-being and education. It gives students two more days of classroom instruction, interactive lessons, and engagement in thought-provoking discussions. Any reduction in testing also means more time with my students, since administrators can pull teachers out of their classrooms for up to a week to score each test.

Still, I know these tests provide us with critical data about how students are doing across our state and where we need to concentrate our resources. The changes address my worries about over-testing, while still ensuring that we have an objective measure of what students have learned across the state.

For those who fear that cutting one-third of the required state testing hours will not provide teachers with enough data to help our students, understand that we assess them before, during, and after each unit of study, along with mid-year tests and quizzes. It is unlikely that one extra day of testing will offer any significant additional insights into our students’ skills.

Also, the fact that we receive students’ state test results months later, at the end of June, means that we are more likely to have a snapshot of where are students were, rather than where they currently are — when it’s too late for us to use the information to help them.

That’s where New York can still do better. Teachers need timely data to tailor their teaching to meet student needs. As New York develops its next generation of tests and academic standards, we must ensure that they are developmentally appropriate. And officials need to continue to emphasize that state tests alone cannot fully assess a student’s knowledge and skills.

For this, parents and teachers must continue to demand that their voices are heard. Until then, thank you, New York Regents, for hearing us and reducing the number of testing days.

In my classroom, I’ll have two extra days to help my special needs students work towards the goals laid out in their individualized education plans. I’ll take it.

Rich Johnson teaches fifth grade at P.S. 105 in the Bronx.

First Person

My high school told me to apply to 100 colleges — and I almost lost myself in the process

PHOTO: Stephanie Snyder

High schools have become obsessed with “million-dollar scholars,” and it’s hurting students.

Across Memphis, students often are pushed by counselors to apply to as many colleges as possible — as many as 100 — all to push students to reach that million-dollar scholarship mark. The more dollars and college acceptance, the better!

I graduated in 2016, and my experience offers a case study.

I’m a pretty well-rounded individual: In high school, I was a finalist in the Let’s Innovate Through Education program and was able to launch SousChef-Memphis, a culinary nonprofit organization. I was a dual-enrollment student and took honors courses. I was committed to community service. I was vice president of my high school organization, Modern Distinctive Ladies. I was on the bowling team, managed the basketball team, and participated in debate forensics and drama.

I was also told by counselors to apply to 100 colleges. I was never told why that number was chosen, but my peers were told the same. We were often pulled out of class to complete these applications, which took away from instructional time — about an hour per day. My high school also ran on an infraction system, and not turning in college applications and other documents led to disciplinary actions.

The quality of those applications only shed a dim light on the student and person that I am. A hundred applications was never my goal. A hundred applications doesn’t measure the capability, intelligence or worth of me as a student. A hundred applications is just ridiculous!

Schools with similar approaches, though, get glowing media coverage. Meanwhile, a lot of that scholarship money is irrelevant, since a single student obviously can only attend one school.

I think that if I had been counseled properly, I would have had a better grasp on my high school-to-college transition. I ultimately chose to leave Memphis to attend another state university on a full scholarship. Looking back, that school was not the best fit for me. I returned to Memphis to attend our local public university.

A counselor never tried to determine what the absolute best school for me would be. I wasted a lot of time, money and resources trying to figure that out. And I almost lost myself in the process.

I was more than capable of getting back on track, and I did. But not every student can afford to go through what I went through.

High schools need to realize that, while students amassing millions of dollars in scholarships and hundreds of college acceptance letters seems like an accomplishment, the outcome for many students is the total opposite.

Too many students end up not going to a school that is the best fit for them, taking on piles of debt, and dropping out with no workforce experience.

The goal should be that each high school student will graduate having a grasp on their career path (and experience in that field), scholarships to the school of their choice (full rides or little to no debt), and be confident in where they will be spending the next four to six years of their life. Being thorough in the college search and submitting quality applications is what leads to a college that is the best fit for the student, obtaining scholarships, and ultimately graduating.

Here’s what I wish a counselor had told me:

"It is time to start thinking about quality over quantity. Quality counseling. Quality applications. And quality choices about what to do next."Anisah Karim

Try things you like, but don’t overload yourself. Look for summer internships that pay, rather than minimum-wage jobs. Build a network of people who can help you make good decisions about college and work. Research schools with a major you’re interested in, and find out what scholarships they offer. Keep an eye on your GPA and make sure you’re taking the classes you need to graduate. Apply for colleges when applications open and submit the FAFSA form in October.

And most importantly, through all four years of high school, don’t be afraid to ask for help.

It is time to start thinking about quality over quantity. Quality counseling. Quality applications. And quality choices about what to do next.

Anisah Karim is a psychology student at the University of Memphis. She plans to continue her education in speech pathology and otology and eventually start her own private practice. She also plans to launch two new business ventures in the fall and relaunch SousChef in the fall of 2018.