First Person

Commentary: Using research correctly

Kristin Klopfenstein is executive director of the Education Innovation Institute at the University of Northern Colorado and a self-proclaimed data geek.

I find it tremendously exciting that high-quality research is cited more and more as the basis for education policy decisions. And I notice that researchers are responding by trying to provide concrete examples of what their results mean, and particularly to show how big their numbers are. It is also becoming apparent that the examples used to explain the size of numbers can have a disproportionate, and often inappropriate, effect on how a particular finding is applied in the policy context.

I’m struck by how dramatically this has happened with the “consecutive great teachers” argument. Over the years, several teams of researchers (Hanushek and Rivkin, Gordon, Kane, and Staiger among them) have reported variations on the theme that achievement gaps could be closed if black or low-income students were assigned several extraordinary teachers in a row. This statement has now been repeated enough in the popular press and education policy circles to become conventional wisdom, with a particularly succinct (and inaccurate) version being: “we now know that three years with a great teacher is enough to close the achievement gap.”

The problem is that when researchers made such statements, they never actually observed a disadvantaged student being exposed to multiple extraordinary teachers over time and reaching the academic level of their more advantaged peers. Rather, researchers calculated the “teacher effect” number by crunching data on a group of students exposed to a variety of good and bad teachers, from a few grades and in a particular subject, and extrapolated this result to other students, grades, and subjects. In other words, for fellow data geeks out there, the consecutive great teacher conclusion is based on an “out of sample” extrapolation.

Suppose the data crunching reveals that a fourth grade math teacher from the top ten percent of all math teachers is shown to increase the average student’s math growth score by some number, let’s say 5. Is 5 big? Small? Somewhere in between? The researcher proceeds to make the case that 5 is big because if a disadvantaged kid gained 5 points every year for 3 years, that would be enough to bring them up to the level of more advantaged students. Readers with research experience tend to take this extrapolation with a grain of salt knowing its limitations. But readers without such experience don’t necessarily have the instincts to realize that this example can’t be taken at face value. And the myth is born.

The question I’m raising today isn’t whether great teachers make a difference. There is credible evidence that they do. The question is whether a student’s gain in one teacher’s classroom can be extrapolated over time to predict a cumulative effect when there were no students in the original research who actually exhibited this result. Kids learn at different rates as they mature and their scores can be affected by many external factors including the type of curriculum used and the motivation levels of other students in the classroom. How likely is it that a student will experience uniform gains year after year working under different teachers? And there’s the issue of basing teacher ratings solely on tests of basic skills, often in just one subject, as many studies do. Can a teacher who improves math scores produce equally large gains in other subjects and, perhaps even more importantly, in the soft skills that employers are crying out for?

Misunderstanding the relationship between teacher quality and other factors influencing the achievement gap can lead to the misallocation of time and money. The consecutive great teacher argument has been used to buttress policy arguments for greater accountability, higher teacher pay, merit pay, and charter schools with liberal hiring and firing practices. None of these policies are inherently good or bad in my view, but in a world of limited resources, a policy focus in one area can limit resources for other areas that might actually be bigger levers for reducing the achievement gap. For example, robust urban planning, housing, and health policies have the potential to significantly decrease student mobility and help kids manage asthma and other health conditions that lead to chronic school absences, other key factors in the achievement gap.

I’m not the first to bring up this issue. Plenty of prominent scholars have raised such questions. Diane Ravitch devotes much of her popular book The Death and Life of the Great American School System to warnings against giving standardized tests too much power. On the issue of gap-closing claims, she quotes Richard Rothstein as noting that “good teachers can raise student achievement, and teachers are defined as good if they raise student achievement” (page 182). A circular argument if there ever was one. So why do the myths survive despite such critiques?

I think one reason magic bullet solutions gain currency is that there is no incentive in academia to write clearly for a general audience and to follow up and make sure research is being applied appropriately.  Tenure and promotion are earned by being cited, not by being cited correctly. Moreover, too few researchers who understand the complex statistics behind the magic bullet solutions publicize their objections broadly, as Ravitch and Rothstein did. It is heady stuff when one’s research enters the public conversation. Correcting misunderstandings in these situations takes courage, especially when truisms are embraced by powerful politicians and policy makers.

First Person

I’m a Bronx teacher, and I see up close what we all lose when undocumented students live with uncertainty

The author at her school.

It was our high school’s first graduation ceremony. Students were laughing as they lined up in front of the auditorium, their families cheering them on as they entered. We were there to celebrate their accomplishments and their futures.

Next to each student’s name on the back of those 2013 graduation programs was the college the student planned to attend in the fall. Two names, however, had noticeable blanks next to them.

But I was especially proud of these two students, whom I’ll call Sofia and Isabella. These young women started high school as English learners and were diagnosed with learning disabilities. Despite these obstacles, I have never seen two students work so hard.

By the time they graduated, they had two of the highest grade point averages in their class. It would have made sense for them to be college-bound. But neither would go to college. Because of their undocumented status, they did not qualify for financial aid, and, without aid, they could not afford it.

During this year’s State of the Union, I listened to President Trump’s nativist rhetoric and I thought of my students and the thousands of others in New York City who are undocumented. President Trump falsely portrayed them as gang members and killers. The truth is, they came to this country before they even understood politics and borders. They grew up in the U.S. They worked hard in school. In this case, they graduated with honors. They want to be doctors and teachers. Why won’t we let them?

Instead, as Trump works to repeal President Obama’s broader efforts to enfranchise these young people, their futures are plagued by uncertainty and fear. A Supreme Court move just last week means that young people enrolled in the Deferred Action for Childhood Arrivals program remain protected but in limbo.

While Trump and the Congress continue to struggle to find compromise on immigration, we have a unique opportunity here in New York State to help Dreamers. Recently, the Governor Cuomo proposed and the state Assembly passed New York’s DREAM Act, which would allow Sofia, Isabella, and their undocumented peers to access financial aid and pursue higher education on equal footing with their documented peers. Republicans in the New York State Senate, however, have refused to take up this bill, arguing that New York state has to prioritize the needs of American-born middle-class families.

This argument baffles me. In high school, Sofia worked hard to excel in math and science in order to become a radiologist. Isabella was so passionate about becoming a special education teacher that she spent her free periods volunteering with students with severe disabilities at the school co-located in our building.

These young people are Americans. True, they may not have been born here, but they have grown up here and seek to build their futures here. They are integral members of our communities.

By not passing the DREAM Act, it feels like lawmakers have decided that some of the young people that graduate from my school do not deserve the opportunity to achieve their dreams. I applaud the governor’s leadership, in partnership with the New York Assembly, to support Dreamers like Sofia and Isabella and I urge Senate Republicans to reconsider their opposition to the bill.

Today, Sofia and Isabella have been forced to find low-wage jobs, and our community and our state are the poorer for it.

Ilona Nanay is a 10th grade global history teacher and wellness coordinator at Mott Hall V in the Bronx. She is also a member of Educators for Excellence – New York.

First Person

I was an attorney representing school districts in contract talks. Here’s why I hope the Supreme Court doesn’t weaken teachers unions.

PHOTO: Creative Commons / supermac1961

Many so-called education reformers argue that collective bargaining — and unions — are obstacles to real change in education. It’s common to hear assertions about how “restrictive” contracts and “recalcitrant” unions put adult interests over children’s.

The underlying message: if union power were minimized and collective bargaining rights weakened or eliminated, school leaders would be able to enact sweeping changes that could disrupt public education’s status quo.

Those that subscribe to this view are eagerly awaiting the Supreme Court’s decision in the case of Janus v. American Federation of State, County, and Municipal Employees. At issue is the constitutionality of “agency” or “fair share” fees — employee payroll deductions that go to local unions, meant to cover the costs of negotiating and implementing a bargaining agreement.

In states that permit agency fees (there are about 20), a teacher may decline to be part of a union but must still pay those fees. If the Supreme Court rules that those agency fees are unconstitutional, and many teachers do not voluntarily pay, local unions will be deprived of resources needed to negotiate and enforce bargaining agreements.

Based on my experience as an attorney representing school districts in bargaining and contract issues, I have this to say to those hoping the Court will strike down these fees: be careful what you wish for.

Eliminating fair share fees (and trying to weaken unions) represents a misguided assumption about bargaining — that the process weakens school quality. To the contrary, strong relationships with unions, built through negotiations, can help create the conditions for student and school success. Indeed, in my experience, the best superintendents and school boards seized bargaining as an opportunity to advance their agenda, and engaged unions as partners whenever possible.

Why, and how, can this work? For one, the process of negotiations provides a forum for school leaders and teachers to hear one another’s concerns and goals. In my experience, this is most effective in districts that adopt “interest-based bargaining,” which encourages problem-solving as starting point for discussions as opposed to viewing bargaining as a zero-sum game.

Interest-based bargaining begins with both sides listing their major concerns and brainstorming solutions. The touchstone for a solution to be adopted in a bargaining agreement: Is the proposal in the best interests of children? This important question, if embedded in the process, forces both sides to carefully consider their shared mission.

For example, some districts I worked with paid teachers less than comparable neighboring districts did. It would have been unreasonable for unions to insist that their pay be increased enough to even that difference out, because that would mean reducing investments in other items of importance to children, like technology or infrastructure. At the same time, it would have been untenable for management to play “hard ball” and deny the problem, because to do so would likely lead to a disgruntled workforce.

Instead, both sides were forced to “own” the issue and collaboratively craft plausible solutions. That made unions more agreeable to proposals that demonstrated some commitment by the district to addressing the issue of pay, and districts open to other things that they could provide without breaking the budget (like more early release days for professional development).

To be sure, many school administrators could get frustrated with the process of bargaining or having to consult the negotiated agreement when they want to make a change. Some districts would very much like to adopt an extended school day, for example, but they know that they must first consult and negotiate such an idea with the union.

Yet, in districts where school administrators had built a reservoir of goodwill through collective bargaining, disagreement does not come at the cost of operating schools efficiently. Both sides come to recognize that while they inevitably will disagree on some things, they can also seek agreement — and often do on high-stakes matters, like teacher evaluations.

How does this relate to the Supreme Court’s pending decision? Without fees from some teachers, unions may lack the resources to ensure that contract negotiations and enforcement are robust and done well. This could create a vicious cycle: teachers who voluntarily pay fees for bargaining in a post-Janus world, assuming the court rules against the unions, will view such payments as not delivering any return on investment. In turn, they will stop contributing voluntarily, further degrading the quality of the union’s services.

Even more troubling, if fair share fees are prohibited, resentment and internal strife will arise between those who continue to pay the fees and those who refuse. This would undercut a primary benefit of bargaining — labor peace and a sense of shared purpose.

Speaking as a parent, this raises a serious concern: who wants to send their child to a school where there is an undercurrent of bitterness between teachers and administrators that will certainly carry over into the classroom?

It is easy to see the appeal of those opposing agency fees. No one wants to see more money going out of their paycheck. The union-as-bogeyman mentality is pervasive. Moreover, in my experience, some teachers (especially the newer ones) do not recognize the hidden benefits to bargaining contracts.

But, obvious or not, agency fees help promote a stable workplace that allows teachers to concentrate on their primary responsibility: their students. Removing the key ingredient threatens this balance.

Mark Paige is a former school teacher and school law attorney who represented school districts in New England. He is currently an associate professor of public policy at the University of Massachusetts – Dartmouth.