human capital

Wide margins of error, instability on city's value-added reports

Some English Language Arts teachers received high "value-added" scores in 2007 but much lower scores in 2008.

The value-added reports meant to measure city teachers’ effectiveness have wide margins of error and give judgments that fluctuate — sometimes wildly — from one year to the next, a new analysis finds.

Schools Chancellor Joel Klein has instructed principals to use the Teacher Data Reports as one way to decide which teachers should receive tenure. Teachers who teach English or math to students in grades three through eight receive the reports.

The NYU economist Sean Corcoran found that 31 percent of English teachers who ranked in the bottom quintile of teachers in 2007 had jumped to one of the top two quintile by 2008. About 23 percent of math teachers made the same jump.

There was an overall correlation between how a teacher scored from one year to the next, and for some teachers, the measurement was more stable. Of the math teachers who ranked in the top quintile in 2007, 40 percent retained that crown in 2008.

The Annenberg Institute for School Reform at Brown University, which has a history of criticizing the Bloomberg administration, published Corcoran’s findings, which were part of a wider look at the practice of assigning “value-added” scores to teachers based on their students’ test scores.

The analysis explains the difference between what value-added scores of teachers aim to do and what value-added measurements actually do in practice. The dream is to isolate the effect of a teacher on students’ performance from the effect of everything else; the reality is that the measures approximate that isolated effect with statistics, weak tests, and small sample sizes.

Corcoran offers some praise. “The simple fact that teachers and principals are receiving regular and timely feedback on their students’ achievement is an accomplishment in and of itself, and it is hard to argue that stimulating conversation around improving student achievement is not a positive thing,” he writes. “But,” he writes,

teachers, policymakers, and school leaders should not be seduced by the elegant simplicity of “value-added.”

The weaknesses of value-added detailed in the report include:

  • the fact that value-added scores are inherently relative, grading teachers on a curve — and thereby rendering the goal of having only high value-added teachers “a technical impossibility,” as Corcoran writes
  • the interference of imperfect state tests, which, when swapped with other assessments, can make a teacher who had looked stellar suddenly look subpar
  • and the challenge of truly eliminating the influence of everything else that happens in a school and a classroom from that “unique contribution” by the teacher

Another challenge for the teachers and principals charged with using value-added scores for self-improvement is the uncertainty about what each individual teacher’s score actually is. On each teacher’s report, the city pinpoints the percentile ranking that represents how she compares to other teachers of the same subject and grade.

But while this is the ranking that the teacher most likely holds, it’s far from 100 percent certain. Indeed, the economists who make value-added scores can only be very certain that the teacher falls somewhere on a range of percentiles (and even getting that cautious, they’re still only 95 percent certain). This range, as you might remember from statistics, is called the “confidence interval.”

For most teachers, the confidence interval is at least 30 percentage points long. For math and English teachers with only one year’s worth of data, the average length is over 60 percentage points. That’s a range of, for instance, between the 10th and 70th percentile of teachers.

The average confidence intervals that Corcoran reports are in the chart below. You can see that, because the confidence intervals shrink as the sample size grows, they are longest when only a year’s worth of data is available.

Teachers in the Bronx face the least certainty. Corcoran guesses that this is because their students are the most likely not to be measured, thereby lowering the data pool — either because the students are classified as special ed or English language learners, and don’t take the state test, or because the students move from year to year, making data about their growth over time harder to come by.

picture-311

The full report is here and below.
The Use of Value-Added Measures of Teacher Effectiveness in Policy and Practice

magnetic fields

Three Indianapolis schools recognized for diversity, but local efforts to integrate are still underway

PHOTO: Alan Petersime
School 27

Three Indianapolis public schools can claim a new title: 2017 National Magnet School of Distinction.

The prize, given annually by a national group promoting the themed schools, recognizes schools that boost student achievement, promote diversity, and have strong community ties. Among this year’s 244 winners nationally are Center for Inquiry Schools 2, 27, and 84, all part of the Indianapolis Public Schools district.

“Being recognized as a Magnet School of Distinction provides just one affirmation to the collective CFI School family that their philosophy, tireless work ethic, community support, and relentless journey to provide students with the absolute best inquiry based education is paying dividends to their students, to IPS, and to the larger community,” said Greg Newlin, the district’s academic improvement officer, in a statement.

The three schools use the International Baccalaureate curriculum. And their students are more likely to be white and more affluent than at the average district school. The schools’ demographics vary widely: School 27 is well integrated, with about 39 percent white students and 41 percent black students. In contrast, School 84 is nearly 83 percent white this year in a district where students of color make up 80 percent of enrollment.

That could soon change. After a series on segregation from Chalkbeat and the Indianapolis Star exposed how rules about magnet school admission gave the most privileged families in the district an edge at sought-after schools, the school board last year voted to adopt policies designed help more low-income students win admission to magnet schools. The new policies could reshape who enters the schools this fall.

“Magnet schools were born out of the civil rights movement and were intended to help school districts to reintegrate,” IPS board member Gayle Cosby said at the time. “We want to make sure that magnet schools are not actually serving a different purpose in our district.”

The award to the Indianapolis schools is the second tier that Magnet Schools of America hands out. Schools that have especially strong academic performance can earn a different title: schools of excellence.

First Person

I’m a teacher, not an activist. Here’s why I’m joining the March for Science this weekend

PHOTO: Creative Commons / Jeremy Wilburn

I became a science teacher because there’s nothing I love more than talking about science. This Saturday, I’ll march for science in Cleveland because there’s nothing I believe is more important than defending science in our society and our classrooms.

My love affair with science goes back to my seventh-grade teacher, Mr. Hurst, who took a hands-on approach to science education. Through labs and real-world investigations, my classmates and I discovered the complexity of scientific discovery. While I originally pursued a career in lab research, I soon realized that my true passion lay in teaching – that I could fulfill my love of science by delivering the same quality of teaching that I’d received to the next generation.

I’m marching for science on Saturday because every student deserves such a strong foundation. A well-rounded education should be a reality for every child in America – and that must include science, technology, engineering and math. Without it, our country won’t be able to solve the very real crises looming just over the horizon.

The world’s population is growing exponentially, consuming a limited supply of natural resources at a faster pace. We rely on nonrenewable forms of energy that we’ll inevitably exhaust at a great environmental cost. Medical advances have slowed the spread of infectious disease, but our overuse of antibiotics is leading to a new generation of drug-resistant pathogens.

Our children need to know what they are up against so they can design their own solutions. They need an education that enables them to think analytically, approach a problem, tackle new challenges, and embrace the unknown. That’s exactly what good science education does.

Still, I understand that some may wonder why teachers are marching – and even if they should. Some will inevitably accuse teachers of “politicizing” science or stepping “out of their lane.”

But marching for science is distinct from the kind of political statements I dutifully avoid in my role as a teacher. To me, marching is a statement of fact: without science teachers, there is no science education; without science education, there is no future for science in America. Science teachers and their classrooms are the agar in the petri dish that cultures our students’ scientific minds. (Did I mention there’s nothing I love talking about more than science?) In any movement for science, teachers have a role to play.

Marching, like teaching, is to take part in something bigger. Years from now, if I’m lucky, I might glimpse the name of one of my former students in the newspaper for a scientific discovery or prestigious award. But by and large, it’s my job to plant seeds of curiosity and discovery in a garden I may never see.

On Saturday, I’ll be there alongside doctors and nurses, engineers and researchers, and citizens from all walks of life who love science and want to see it valued and respected in our country. We might not see the fruit of our labors the day after the march, or even after that, but the message we send will be clear.

If you’re a parent or student – maybe one of my own – I hope you see that passion for science on full display around the nation this Saturday. I hope you see why having committed science teachers like myself and my colleagues is inextricably bound to the fate of our world. I hope that recognition grows into action to support teachers and demand universal access to an excellent science education, like the one I strive to provide every day in my classroom.

Sarah Rivera teaches engineering, biology, biomedical science, and environmental science at Perry High School in Perry, Ohio. She is also a member of 100Kin10’s teacher forum.