Commentary: Diving deep into value-added study

Robert Reichardt, the former director of the Center for Education Policy Analysis at CU-Denver’s School of Public Affairs, is president of R-Squared Research, LLC, a local research firm.

A lot of attention has been paid to the recent work on the long term impacts of teachers by Chetty, Freidman, and Rockoff.  The New York Times, EdNews Colorado, and Mother Jones all discussed this remarkable research.   The researchers were able to show a correlation between value-added measures of teacher effectiveness in English and math for Grades 3 through 8 and:

  • Reduced teen pregnancy rates (with larger reductions for minority and low income students).
  • Increased college attendance rates.
  • Increased quality of colleges attended (as measured by earnings of the college’s graduates).
  • Increased lifetime earnings.

They also found that, as would be expected, movement of an effective teacher out of a school reduced the average growth of students in that school and movement of an effective teacher into a school increased the average growth of students in that school.

Finally, they showed that the removal of a low-performing teacher (bottom 5 percent of teachers) and the replacement of  that teacher with an average teacher would increase the average lifetime earnings of a student in that  class by $9,422.

The report is dense: Almost 100 pages of economist-speak. However, sprinkled throughout the analysis are additional interesting tidbits that have useful policy implications in Colorado beyond reinforcing the value of effective teachers. I highlight a few of these findings here.

A key policy question for Colorado and the nation is how to identify effective and ineffective teachers.  These authors, like many others, say that multiple measures should be used to identify effective teachers including principal evaluations and classroom observations.  They also provide an estimate of the error-rate when using value-added measures to identify low performing teachers.  They estimate that value-added measures using three years of data would identify low performing teachers about 70 percent of the time (i.e. an error rate of 30 percent).

One of the policy questions they try to address is whether districts should use bonuses to retain effective teachers instead of removing lower performing teachers. Their analysis suggests that a general bonus to effective teachers would not be cost-effective because many effective teachers would return without the bonuses. In other words, they recommend focusing policy on removing ineffective teachers more than on using bonuses to retain effective teachers.

When looking at the impact on lifetime earnings from effective teachers, they found this impact is larger for students from higher-income families.  The data suggests that high-income families were better able to translate higher test scores into attendance at a higher quality college.  In other words, for families and society to fully benefit from investment in teacher quality, we need to also support policies that provide equal access to quality higher education for all students.

A key struggle when interpreting test scores is that we know teachers and schools teach more than academic knowledge. Kids also learn social skills, cooperation, creativity, citizenship, and a whole host of other important behaviors, which are important in life but not directly measured in math and science scores.

The outcome measures used in this study, particularly teen pregnancy and income suggest that while good teachers help kids learn academic subjects, they also help kids learn these other important life skills.

One worry is that value-added measurement is that it mixes the actual impact of teachers with tracking of kids from well-off families into particular classrooms.  These researchers were able to use the tax records to show that their value-added measurements are not a product of some teachers getting the higher-income students and value-added does measure a long-term impact of teachers.  The correlation of value-added measures of teacher effectiveness with longer term outcomes can be seen as a validation of value-added measures.

Key to this impressive research is the availability of longitudinal data that links teachers, students and in the case of this research federal tax records.  This reinforces the value of longitudinal data sets being built at CDE, CDHE, and at many districts.  This research supports many of the policy efforts in Colorado:

  • Improving teacher effectiveness
  • Developing longitudinal data systems
  • Increasing access to college for all students

There is a lot of work to be done, but in many areas we are focusing on the right things.

About our First Person series:

First Person is where Chalkbeat features personal essays by educators, students, parents, and others trying to improve public education. Read our submission guidelines here.