Monday afternoon, I had the opportunity to respond to Merryl Tisch, Chancellor of the Board of Regents, and David Steiner, the New York State Commissioner of Education, as they talked about the future of P-16 education in New York State at the Phyllis L. Kossoff Policy Lecture at Teachers College, Columbia University. I wasn’t sure what they’d say, so prepared some remarks responding to the proposals regarding teacher education in New York State that the Commissioner presented to the Board of Regents a few weeks ago. For the handful of readers who might be interested, here’s what I wrote. (Due to time constraints, I didn’t say all of this at the event.) Chancellor Tisch and Commissioner Steiner were quite willing to hear and engage with the critiques that my colleague Lin Goodwin and I offered, and I look forward to continuing this conversation with them.
It’s no surprise that the State Education Department and the Board of Regents have taken up the cause of ensuring an equitable distribution of highly-qualified teachers across New York State. The key justification for such a goal is the fact that the K-12 education system is shortchanging our children. Although some students are highly successful, many more are not, and the problems are concentrated in urban school systems serving large numbers of poor children of color.
If that’s the problem, is improving the education of teachers the solution? It’s certainly part of the solution, given what we know about the centrality of teaching to student learning. But it’s by no means the entire solution, as a great many other forces shape student outcomes. For example, a great teacher can’t compensate for a child coming to school hungry, and great teaching of an out-of-date curriculum only results in great mastery of out-of-date knowledge. I trust that Chancellor Tisch and Commissioner Steiner are not seduced by claims that the single most important determinant of a child’s achievement is the quality of his or her teachers, because that’s simply not true. Family background continues to be the dominant factor. But the quality of teachers is, at least in theory, something that is manipulable via education policy initiatives, and it’s a lot more tractable than addressing the fact that one in five children under the age of 18 in New York State live below the poverty line.
So if we redefine the problem as too few students in New York State are taught by highly qualified teachers, then what’s the solution? The recent proposal brought by the State Education Department to the Board of Regents offers five recommendations. In some ways, this package of proposals is a straightforward extension of trends that have shaped the course of public education in the U.S. over the past two decades: a heightened concern for holding school systems, schools and teachers accountable for student outcomes, with more oversight by the federal government and the state; an expansion of the role of markets in the operation of public schooling, coupled with the belief that market forces will reward successful enterprises, and drive the unsuccessful out of business; and a targeted deregulation that allows some institutions to strike a bargain for increased autonomy in exchange for increased accountability to the state.
There’s not a whole lot of evidence that these kinds of policy reforms have led to better educational outcomes for children in the U.S. It’s challenging to show that accountability systems such as No Child Left Behind or state-level initiatives have led to substantially higher achievement; and the charter schools spawned by the expansion of markets have, by most accounts, included some great successes and some abject failures, with little overall impact on student performance across the country. So I am not optimistic that the proposed reforms will result in sharp improvements in teaching and learning in public schools in New York State.
As is often the case with public policy, the devil is in the details, and I’d like to take a couple of minutes to mention two things that I’m worried about. The first is the heightened attention to classroom effectiveness in both performance-based assessments for teacher certification, and in the assessment of the institutions which offer teacher preparation programs. I worry that the State Education Department and the Board of Regents will be obliged to rely on an overly-narrow set of measures of classroom effectiveness, due to the constraints of time, money, and technology.
Portfolio assessments, including lesson plans, videotapes of teaching practice, collections of student work, and candidate self-assessments are a promising direction, and there are some existing models that may be useful to guide the design of such assessments. I’d be happier if there were more evidence that the implementation of portfolio assessments resulted in better teaching and learning, and if there were a more explicit theory of how having assessments produces better teaching.
But it’s striking that the recommendations single out value-added student assessment data as components of both the portfolios of candidates for professional certification and of the profiles of certifying institutions. Simply put, the technology for using value-added student assessment data for these purposes is not ready for prime time, and likely will not be for many years to come.
One major obstacle is the lack of reliable and valid measures of student performance that can serve as the basis for value-added assessments of teacher effectiveness. When the 2009 New York State results from the National Assessment of Educational Progress were released last month, they showed a flat level of performance at odds with the sharp growth observed on the New York State exams at every grade level in English Language Arts and mathematics over the past several years. Commissioner Steiner and Chancellor Tisch have both expressed concern that the state’s testing system is not telling us what we need to know about the academic performance of New York State’s children and youth. The tests and test items have become predictable over time, and some grade-level standards have never appeared on the exams. There are, for example, 48 eighth-grade math standards. This year, just seven standards made up 50% of the points on the eighth-grade exam.
The state assessment system is broken, and it can’t be fixed overnight. I strongly urge the Commissioner and Chancellor to take the time to develop a new assessment system that more accurately measures students’ mastery of school subjects, and that is designed to measure changes in students’ learning over time. And, if the objective is to use the assessment system to evaluate teachers, schools, school districts, and institutions of higher education, it’s important to build that objective into the design of the assessments, rather than using them for a purpose for which they were never designed in the first place.
The technology for test design is surprisingly complex and fragile, and we do ourselves no favors by assuming that we can just whip up a test overnight.
The Regents exams, which I assume would be central to measures of value-added effectiveness at the secondary level, are problematic in their own right. I don’t understand why the State Education Department has high school students’ Regents exams scored by the teachers at their own schools. One big idea that the Freakonomics school probably has right is that people respond to incentives. And there are powerful incentives for schools to maximize students’ scores on Regents exams in ways that compromise the validity of those scores. We see the fruits of this when students go off to college and find that they are unprepared for college-level work.
So in essence, I’m saying to Commissioner Steiner and Chancellor Tisch, “Clean up the state assessment system — and take the time to do it right. Then we can talk about value-added assessment.”
But beyond the many questions about value-added effects on students’ test scores, we should be asking, how do we assess a teacher’s contributions to other learning outcomes? Surely we care about more than test scores. What are good measures of a teacher’s contributions to preparing students to be competent citizens in our democracy? How much are the Board of Regents and the State Education Department willing to invest in creating measures that will capture how well teachers teach students to think, question and act?
A brief vignette may reveal the challenge. It’s January, and Ms. Bilsky, a fourth-grade teacher in the Bronx, is teaching a math lesson. The subject is geometry, and the lesson is about how to classify angles as either acute or obtuse. The topic is a standard from the state’s math core curriculum. In the middle of the lesson, Rashid, a boy in the class, audibly aims a racial slur at his classmate Javier. Ms. Bilsky hears it, but she chooses to ignore it, instead plowing ahead with the lesson. At the end of the year, the students in Ms. Bilsky’s class did a bit better on the state math assessment than the students in other fourth-grade classrooms in the Bronx.
Now, is that good teaching?
The value-added assessment will tell us that it is good teaching.
And a teacher preparation program sending a cohort of Ms. Bilsky’s out into the field might look pretty good too. But I think we should demand more of our teachers and our teacher preparation programs than simply raising students’ test scores. And I think we should demand more of the New York State Education Department in developing measures that can capture a broader array of outcomes of good teaching. I do not doubt that it will be difficult to do so; important things often are.
The second issue I want to discuss is the proposal to pilot a new teacher certification model that would enable providers other than institutions of higher education to offer teacher preparation programs, with the Board of Regents awarding Master’s degrees to the candidates who complete these programs. I am deeply troubled by this prospect, because it seems to be a serious threat to the very nature of graduate education. Currently, state regulations require that most graduate-level courses in certification programs be offered by full-time faculty holding terminal degrees, with the assumption that these faculty are scholars of the subjects they teach. Now, we all know that there are schools of education, and other professional schools, around the state, where the quality of research may not be very high, and the contribution to a body of knowledge about real-world practice may not be very great. But the explicit decoupling of the production of knowledge from the preparation of practitioners is, in my view, a very bad idea. And one might wonder whether other occupations regulated by the state will be far behind. Will health clinics be authorized to prepare physicians? Will mental health facilities train psychologists?
Moreover, I suspect that a proposal such as this might tax the capacity of the State Education Department to offer appropriate oversight. Would the Department have to devote staff to ensure that these pilot programs met existing criteria for preparation — libraries, facilities and physical space, qualified faculty, relationships with local schools and field placements, curricula and syllabi, and so on? Or would these pilot programs not have to meet the state’s existing standards?
I think what’s most baffling in this proposal is the awarding of a Master’s degree by the Board of Regents. Why is it necessary for the state’s recognition of completion of a teacher preparation program to be coupled with a Master’s degree? Neither No Child Left Behind nor the Individuals with Disabilities Education Act requires that a highly-qualified teacher have a master’s degree, and there are plenty of undergraduate education programs around the state of New York that lead to teacher certification. Having the Board of Regents award degrees that are widely understood to be the province of accredited institutions of higher education in a sense turns the State Education Department into a giant ed school. Doesn’t the Department have enough problems, without tarring it with that particular brush?