roland fryer returns

Study: $75M teacher pay initiative did not improve achievement

New York City’s heralded $75 million experiment in teacher incentive pay — deemed “transcendent” when it was announced in 2007 — did not increase student achievement at all, a new study by the Harvard economist Roland Fryer concludes.

“If anything,” Fryer writes of schools that participated in the program, “student achievement declined.” Fryer and his team used state math and English test scores as the main indicator of academic achievement.

Schools could distribute the bonus money based on individual teachers' results, but most did not. Most teachers received the average bonus of $3,000.

The program, which was first funded by private foundations and then by taxpayer dollars, also had no impact on teacher behaviors that researchers measured. These included whether teachers stayed at their schools or in the city school district and how teachers described their job satisfaction and school quality in a survey.

The program had only a “negligible” effect on a list of other measures that includes student attendance, behavioral problems, Regents exam scores, and high school graduation rates, the study found.

The experiment targeted 200 high-need schools and 20,000 teachers between the 2007-2008 and 2009-2010 school years. The Bloomberg administration quietly discontinued it last year, turning back on the mayor’s early vow to expand the program quickly.

The program handed out bonuses based on the schools’ results on the city’s progress report cards. The report cards grade schools based primarily on how much progress they make in improving students’ state test scores. A so-called “compensation team” at each school decided how to distribute the money — a maximum of $3,000 per teachers union member, if the school completely met its target, and $1,500 per union member if the school improved its report card score by 75%.

The deal was seen as a landmark in 2007 when Mayor Bloomberg announced it with then-United Federation of Teachers President Randi Weingarten by his side. “I am a capitalist, and I am in favor of incentives for individual people,” Bloomberg said then, while Weingarten emphasized that schools could decide to distribute bonuses evenly among educators. She called the program “transcendent.”

In his study, published as a National Bureau of Economic Research working paper, Fryer writes that researchers were surprised to see that schools that won bonuses overwhelmingly decided to distribute the cash fairly evenly among teachers. More than 80 percent of schools that won bonuses gave the same dollar amount to almost all of the eligible educators.

Researchers were also surprised to find that middle school students actually seemed to be worse off. After three years attending schools involved in the project, middle school students’ math and English test scores declined by a statistically significant amount compared to students attending similar schools that were not part of the project.

The study adds to a research literature on teacher incentive pay that is decidedly more lukewarm than much of the popular conversation about teacher pay. Fryer, himself a strong early advocate of experimenting with financial incentives to improve student achievement, calls the literature “ambivalent.” While programs in developing countries such as India and Kenya have had positive effects, few teacher incentive pay efforts in the United States have been deemed effective.

Almost all schools gave nearly all of their teachers the same sized bonus.

Nevertheless, a person’s position on teacher merit pay has become a litmus test for her reform credentials in many education circles. During his campaign, President Obama used his support for merit pay — traditionally scorned by teachers unions — as evidence that he was willing to challenge traditional Democratic Party thinking. Now, the Obama administration has boosted support for the Teacher Incentive Fund, a program that funds local experiments in incentive pay.

What explains the discrepancy between programs in the U.S. and elsewhere? Fryer rejects several explanations. He argues that the $3,000 bonus (just 4 percent of the average annual teacher salary in the program) was not too small to make a difference, citing examples of effective programs in India and Kenya that gave out bonuses that were an even smaller proportion of teachers’ salaries. He also rejects the possibility that schools’ decisions to use group, rather than individual, incentives was the problem, citing a 2002 study of a program in Israel that used group incentives.

Instead, he says the challenge is that American plans aren’t clear about what teachers can do to receive the reward. In New York City, the bonuses didn’t come simply if students’ test scores rose; the test scores had to rise in comparison to a group of similar schools. So did other measures considered by the city report card, including the surveys that ask students, teachers, and parents for subjective opinions about schools.

Fryer argues that the complexity made it “difficult, if not impossible, for teachers to know how much effort they should exert or how that effort influences student achievement.”

To and Through

Newark’s post-grad paradox: More students are entering college, but few earn degrees

PHOTO: Patrick Wall/Chalkbeat
Newark Mayor Ras Baraka wants 25 percent of residents to have college degrees by 2025, up from 19 percent today.

When it comes to college, Newark faces a good news-bad news paradox.

More students than ever are graduating high school and enrolling in college, according to a new report. Yet fewer than one in four Newark students earns a college degree within six years of graduating high school — leaving many with limited job prospects in a city where an estimated one-third of jobs require a four-year college degree.

Now, city officials are promising to build on the report. They want to ramp up the rigor of high-school classes and create more early-college programs to increase the odds of students entering college and leaving with a degree.  

“How do we teach our children to perform — to graduate?” Mayor Ras Baraka asked at a press conference Wednesday to mark the official release of the report of Newark students’ college outcomes. “We got them in the door,” he said of students who attend college. “Now how do we make them stay?”

The city’s plans, to which Superintendent Roger León is lending his support, reflect a growing recognition that simply getting students into college is not sufficient — and can even backfire if they drop out before graduation, leaving them with college debt but no degree.

Until recently, the charge given to high schools in Newark and across the country was to foster “college-going cultures.” And these efforts showed promising results: On average, 51 percent of Newark Public School students who graduated high school between 2011 and 2016 immediately enrolled in college, up from 39 percent who did so between 2004 and 2010, according to the report by the Newark City of Learning Collaborative, or NCLC, and Rutgers University-Newark’s School of Public Affairs and Administration.

But entering college didn’t guarantee its completion. Of those students who started college straight after high school, only 39 percent earned a degree within six years, the report found.

As a result, educators and policymakers have begun to think harder about how to help students “to and through” college — to ensure they actually earn degrees. Toward that end, Baraka and the NCLC — which includes roughly 40 colleges, schools, nonprofits, and corporations — has set a goal of 25 percent of Newark residents earning college degrees or comparable credentials by 2025.

Today, just 19 percent of Newark adults have associate degrees or higher — compared to 45 percent of adults across New Jersey and 40 percent nationally.

Superintendent León, who began overseeing the city’s schools on July 1, said his main strategy for supporting these efforts will be to expose students to challenging work early on.

“If we don’t do something dramatically in classrooms to improve instruction and make it rigorous,” León said after Wednesday’s event, then students are “getting into college but they’re not completing it.”

Source: “Post-Secondary Outcomes of Newark High School Graduates (2011-2016)” report. Note: The four-year rate is an average of the classes of 2011 to 2013. The six-year rate is from the class of 2011. Graphic: Sam Park/Chalkbeat

For starters, León said he wants high schools to offer more college-level classes. In the 2016-17 school year, just 21 percent of Newark students were enrolled in one or more Advanced Placement or International Baccalaureate classes — compared to 42 percent of students statewide.

He also vowed to raise the quality of instruction in the district’s traditional high schools. Only 14 percent of their graduates earn college degrees within six years, compared to 42 percent of graduates from the city’s selective magnet schools, the report found.

To do that, León said he will create specialized academies within the traditional schools modeled on the magnets, which have specialized themes such as science, technology, or the arts. The academies, which will partner with colleges, will most likely feature admissions criteria similar to those of magnet schools, which select students based on their academic and attendance records, León added.

And, for the first time, all ninth-grade students this academic year will take the Preliminary SAT, or PSAT, León said Wednesday. An additional 1,100 eighth-graders who passed at least one of their seventh-grade PARCC exams will also take the PSAT when it’s administered on Oct. 10.

Since 2016, the district has provided the PSAT to all 10th and 11th-grade students. But León said that giving the test to younger students will focus their attention on college and help identity those who are ready for advanced classes. The PSAT is designed to help students prepare for the SAT, which is used in college admissions, and to qualify for National Merit Scholarships.

The district, which was under state control for 22 years until February, is getting some assistance in its effort to improve students’ college outcomes.

For instance, KIPP, the national charter-school network with eight schools in Newark, is sharing its strategies for helping students choose the right college with guidance counselors at three district high schools.

And the higher-education institutions in the Newark City of Learning Collaborative, including Essex County College and Rutgers University-Newark, plan to create more “dual-enrollment” programs that allow high-school students to earn college credits, said NCLC Executive Director Reginald Lewis.

“We’re all going to do a better job,” Lewis said, “of making sure that once Newark residents get in our doors, that we help them persist.”

Time crunch

In victory for teachers union, Newark superintendent scraps longer hours for low-performing schools

PHOTO: Patrick Wall/Chalkbeat
Superintendent Roger León at Hawkins Street School, one of the schools that will lose its extended hours.

Newark’s new superintendent is eliminating a program that extended the hours of struggling schools, which the teachers union has long attacked as ineffective and unfair to educators.

Teachers at roughly 30 schools will no longer receive $3,000 annual stipends for the extra hours, a provision written into the current teachers contract, which extends to 2019. Instead, all 64 district schools will get extra funding for before and after-school programs, Superintendent Roger León said in an email to employees on Tuesday.

The changes will go into effect Monday, Sept. 10, resulting in new hours for the affected schools just days after the new school year began. The district is still working to adjust pickup times for students who are bused to school, according to León’s email. A few of the schools will phase out their extended hours later in the year, the email said.

“We will not continue to do the same things as before and be surprised when the results do not change,” León wrote, adding that cutting the extra hours would save the district $5 million.

In an interview with Chalkbeat Thursday, León said the move is intended to create more uniformity among schools and the services they provide. Now, all schools will get additional money to pay for programs outside of the regular school day, which schools can tailor to their individual needs, though students who are struggling academically will continue to receive “intensive” support, he said.

“Ultimately, the idea would be by October having completely different after-school and before-school programming that meets the needs of each respective school,” León said.

The extended time was first included in the teachers contract in 2012 as part of a larger improvement plan for the targeted schools, which was developed by Cami Anderson, Newark’s former state-appointed superintendent. The plan also designated some low-performing schools as “renew” schools, where teachers had to reapply for their positions and work longer hours.

Anderson also closed some schools and gave principals new hiring authority. Both actions left dozens of tenured teachers without positions, so Anderson created a fund to pay those teachers to perform support duties in schools. In 2014, that fund for “employees without placement” cost the district $35 million out of its nearly $1 billion budget, though by last year the fund had shrunk to $8 million for about 100 unassigned teachers, according to officials.

León said in Tuesday’s email that he was also eliminating the fund, which he said would save the district another $6 million. The teachers union president said he believed all the unassigned teachers now have placements, but the district did not respond to a request to confirm that.

León is also removing the “renew” and “turnaround” labels from low-performing schools, citing their “progress and student achievement,” according to the email.

“I applaud everyone’s efforts at renew or turnaround schools and acknowledge what has been accomplished,” he wrote.

Now that León has abolished his predecessors’ school-improvement program, he will be expected to create his own. Many schools remain mired in poor performance, even as the district overall has made strides in recent years.

When the teachers union agreed to the extended hours in its 2012 contract with the district, it was hailed nationally as a major breakthrough in efforts to revamp troubled schools. But even as the union agreed last year to keep the provision in its current contract, union officials have assailed the turnaround effort as a failure.

NTU President John Abeigon told Chalkbeat on Thursday that the program had been a “scam” and “nothing more than extended childcare.” He added that the stipend teachers received amounted to about $7 per hour for the extra time they worked.

In 2016, a district-commissioned survey of 787 teachers at schools with extended hours found that two-thirds of teachers at schools where the extra time was spent on student instruction said the time was valuable. But in a survey the union conducted in April, the 278 teachers who responded gave the extended hours low ratings for effectiveness in boosting student achievement.

Some teachers in the union survey praised the longer hours, saying their schools used them effectively to lengthen class periods, run after-school clubs, or allow teachers to plan lessons or review student data. But others said the extra time was squandered, leaving staff and students exhausted with little evidence of improved student outcomes to show for it. (Students’ pass rates on state tests stayed flat or declined at most “renew” schools in the first years of the program.)

The union also has complained that many teachers felt compelled to work the extra hours because those who refused to could be transferred to different schools. Under the terms of the original extended-day agreement, teachers were required to work an extra hour per day and attend trainings during the summer and some weekends.

In León’s email to employees, he said every extended-day school had set different work requirements and “none are consistent with the original design.” The longer days may also be contributing to high teacher turnover in those schools, he wrote, adding that principals of schools with regular hours told him they did not want to extend their hours.

Abeigon, the union president, applauded León’s decision to scrap the extra work hours.

“He came to the conclusion that we expected any true educator to reach: that the program was not working and was never going to work,” he said.

León said Thursday that he is now working on a new turnaround program. Once it’s ready, he promised to share the details with affected families before publicly announcing which schools are part of it — an effort to avoid the student protests that erupted when Anderson identified her “turnaround” schools.

He also said he was still considering whether he would ever close schools that fail to improve or to reverse their declining enrollments. Anderson’s decision to shutter nearly a dozen long-struggling schools continues to fuel resentment among her critics even years later.

“I think the whole idea of how much time does a school get to correct itself is a very important one and I’m going to need to be really reflective on it,” León said. “I’ve seen what closing schools does with people who do not feel that they were aware of it or a part of fixing it.”