More than scores

New research takes an in-depth look at Chicago charter schools — and finds good news beyond test scores

PHOTO: Creative Commons / Charles Wiriawan

Most of what we know about the effectiveness of charter schools has come from scrutinizing testing data. That’s frustrated some who see the focus on test scores as too narrow. Now, a major study recently released by the University of Chicago Consortium on School Research joins a growing body of research that moves beyond tests to judge charters.

This analysis focuses on charter high schools in Chicago, which now educate over a fifth of Chicago high school students. The main takeaways: charter schools seem to help students in the short- and long-run, but those schools also have higher student turnover.

“Test scores are important, but so are other things,” said Julia Gwynne, one of the study’s authors.

The study’s findings will provide grist for both supporters and critics of charter schools. It also offers a number of important takeaways for national charter school observers.

The study shows that one of the larger charter high school sectors in the country is having big positive effects on students.

Attending a charter high school in Chicago led to substantial improvements in test scores, high school attendance, college enrollment, and college persistence.

These effects were relatively big: charter students were in attendance about eight more days on average and scored a full point higher on the ACT (which is out of 36 points). They were nearly 20 percentage points more likely to enroll in a four-year college, and also much more likely to persist in college through four semesters.

“The college enrollment rates are quite high – I would put that at a very sizable difference,” said Gwynne.

Source: UChicago Consortium of School Research

(It’s worth pausing here to note that although charters had a big edge in getting students to college, the college persistence for both sets of students was fairly low, echoing a national trend.)

These are not just raw comparisons of performance: the authors control for a variety of factors that might affect student performance in high school, including poverty, eighth grade test scores and attendance rates, and special education status. This strongly suggests that differences between students are caused by the quality of their schools. The study largely uses data between 2010 and 2013, although the paper goes further back to assess college outcomes, focusing on students who entered high school between 2008 and 2010.

These results are consistent with another recent study showing that Noble, a large charter network in Chicago, led to a big boost in college attendance.

On one important measure charter students were on par with district kids: high school graduation rates were essentially identical.

Interestingly, the results from this study are quite similar to research on Boston’s charter schools. Students there saw big gains in test scores and four-year college enrollment, but the same or lower high school graduation rates.

It’s among the first studies to document a common criticism lobbed at charter schools: high rates of student attrition.

There’s a big caveat to these findings: students are much more likely to leave a charter high school than a district one.

The natural question when considering the two main findings from the study — better outcomes but higher transfer rates — is whether one causes the other. That is, do Chicago charters just get better results because only more successful kids stick around? No, at least not directly.

To measure the effects of charters, the study follows students over time. Charter schools are judged by the performance of all students who start ninth grade at a charter — even if they subsequently transfer out. This is a common technique among researchers and means that the estimated charter impacts are likely conservative.

Still, the paper adds weight to a criticism that has dogged charters across the country: that they push kids out who are struggling or who have behavior problems. There have been a number of anecdotes to support this, but previous studies in a few districts, including New York City and Denver, had generally found little evidence that students leave charters at high rates.

Generally, low-achieving students in Chicago charters were more likely to transfer out than high-achieving ones — but that pattern was also true in district schools. Charter high schools had high attrition rates among both high- and low-performing kids.

The research does not show why students were more likely to leave charter schools.

Source: UChicago Consortium of School Research

The study highlights that even in a successful charter sector, there are some really low-performing schools.

It’s often heard in the charter school debate: charter success varies significantly from school to school. This is hardly surprising; it’s true of district schools too. But the Chicago study finds that on measures like test scores and college enrollment, charters vary even more than district schools in their impact on students.

That means there are more really good charter schools, but also more bad ones, even as on average the charters are better. (There is some evidence that is true of charter schools in Arizona, North Carolina, and Texas.)

Source: UChicago Consortium of School Research

The findings suggests that policymakers should be wary of judging a school by its sector, and that they need to be especially vigilant for struggling charter schools.

The study hints at — though doesn’t definitively explain — why some charter schools are successful.

The latest research can’t show why, overall, Chicago’s charter high schools seem to be high-performing, but it can point to ways they differ from district schools.

Charter high school teachers reported a higher sense of trust and collective responsibility among colleagues. Charter students said their schools were more engaged in helping them plan for life after high school, and teachers said there were greater expectations for college attendance. The schools also had tougher requirements for moving on to the next grade and for graduating high school.

Perhaps surprisingly, charters had a similar number of school days as the district.

Charter schools may have certain advantages over district schools. For instance, charter teachers report that parents are significantly more involved in schools; this may be a reflection of how charters work with parents, but it could also be about the families who select charters (or both).

The fact that charter schools have higher transfer rates may also matter, as does the finding that those students usually end up in district schools. Other research has shown that students entering school mid-year can hurt the performance of their peers, which might be a particular challenge for Chicago’s neighborhood schools.

The study does not look at financial differences across the sectors, including the role of outside money in supporting charter schools, nor does it examine school discipline or student suspension rates.

“What our next steps should be — really understanding best practices in high-performing charter schools is probably at the top of the list,” said Gwynne, the researcher. “To really understand what’s happening on the ground, you need to send researchers into schools.”

beyond high school

Report: Memphis students from poor families less likely to have access to advanced coursework

PHOTO: By Glenn Asakawa/The Denver Post via Getty Images

While most high school students in Tennessee’s largest district have access to advanced courses to prepare them for college, most of those classes are concentrated in schools with more affluent families.

Of the 14 high schools in Shelby County Schools that offer more than 40 advanced classes, all but one have a lower percentage of students from poor families than the district.

Those schools educate slightly more than half of high school students in the Memphis district. In contrast, about a quarter of high school students are in schools with 20 or fewer advanced courses, according to a new district report.

District officials say those course offerings in the 2017-18 school year are closely correlated with the size of the school: The larger the student population, the more likely the school is to offer advanced courses. The concentration of schools with more affluent students was not examined in the report.

PHOTO: Shelby County Schools

The findings are scheduled to be presented at next week’s school board meeting as part of the district’s monthly check-in on various statistics on teaching and student learning.

Taking advanced classes in high school introduces students to college-level coursework and in many cases allows them to skip some college classes — saving students thousands of dollars. And because students from low-income families, who make up about 59 percent of Shelby County Schools, lag behind their more affluent peers in college enrollment, they are encouraged to take more advanced courses.

Advanced courses include programs such as such as Advanced Placement, dual enrollment, International Baccalaureate, and honors courses.

Jessica Lotz, the district’s director of performance management who compiled the report, said this year’s numbers are better than last year. Since her last report on the topic, three schools now offer advanced courses for the first time.

Staffing is the biggest barrier to offering more advanced courses, she said. So, additional teacher trainings are planned for the summer.

And district plans are underway to increase the number of students taking those courses. The district is also pursuing federal funds to help students from low-income families pay for dual enrollment courses, and also encouraging area colleges to lower the number of students needed to take a class so that smaller schools can participate.

The number of students taking advanced courses is part of the state Department of Education measure of a being ready for college, or a “ready graduate,” under its new accountability plan.

Scroll down to the bottom of this story for a full chart on the number of advanced courses by high school.

Here are the 14 schools with 40 or more advanced courses each:

  • White Station High (143 advanced courses)
  • Central High (116)
  • Middle College High (98)
  • Germantown High (95)
  • Cordova High (79)
  • Overton High (75)
  • Ridgeway High (74)
  • Bolton High (56)
  • Southwind High (55)
  • Whitehaven High (52)
  • Hollis F. Price Middle College High (46)
  • Kingsbury High (45)
  • Memphis Virtual School (43)
  • East High (42)

Note: The number of courses offered refers to unique advanced courses that are available at a given school, not the total number of times/sections the same course is offered for different groups of students.

Four high schools did not offer any advanced courses: Legacy Leadership Academy, a charter school; The Excel Center, an adult learning school; Newcomer International Center, a new high school program for immigrant students; and Northwest Prep Academy, an alternative school.

Of the advanced courses, International Baccalaureate, a high-profile certification program for high school students worldwide, was the least common. Just three more affluent high schools — Ridgeway, Germantown, and Bolton — offered those courses, according to the district’s data.

Dual enrollment, another category of advanced courses, are taught in partnership with an area college and count toward a postsecondary degree. Though the share of Shelby County Schools students taking dual enrollment courses has increased from about 5 to 9 percent since 2014, the percentage slightly decreased this year compared to last school year.

Most of the high schools, offer a total of 183 dual enrollment courses. But only four of the 16 charter schools in the report offered those classes.

About half of high schools in the district offer a total of 194 Advanced Placement courses, which culminate in a test at the end of the year that can count toward college credit if students score well enough. Most of those classes are concentrated in seven more affluent schools.

Those schools are:

  • White Station High (39 AP courses)
  • Central High (20)
  • Cordova High (15)
  • Kingsbury High (13)
  • Overton High (13)
  • Whitehaven High (11)
  • Southwind High (10)

Honors courses, which count toward an advanced high school diploma but do not count for college credit, were the most common with just over 1,000 across the district. Only seven schools, which were either charter schools or alternative schools, did not offer any honors courses.

One of Shelby County Schools’ goals is to increase the percentage of students prepared for college by 2025. Currently, about 90 percent of students who graduate from the district would be required to take remedial classes in college because of low ACT scores, according to state data. That’s usually a sign that their high school did not adequately prepare them for college classes.

A state report released last fall examining where students go after high school showed that 56 percent of Shelby County Schools’ graduating class of 2016 went on to enroll in a four-year college or university, community college, or technical college. That’s compared to 63 percent of students statewide.

One of the report’s recommendations to boost that number was to improve partnerships with universities and increase the number of advanced course offerings — a recommendation Lotz emphasized Tuesday.

Shelby County Schools partners with the following universities and colleges for dual enrollment courses: Bethel University, Christian Brothers University, LeMoyne Owen College, Southwest Tennessee Community College, Tennessee College of Applied Technology, University of Memphis, and William Moore College of Technology (Moore Tech)

Below you can find the advanced course offerings at each district-run and charter school in Shelby County Schools. Below that you can view the district’s full report.

New research

From an ‘F’ to an ‘A’, Tennessee now sets high expectations for students, says Harvard study

PHOTO: Lisegagne/Getty Images

Criticized for setting low expectations for students just a decade ago, Tennessee has dramatically raised the bar for standards that now rank among the top in the nation, according to a new analysis from Harvard University.

The state earned an “A” for its 2017 proficiency standards in a study released Tuesday by the same researchers who gave Tennessee an “F” in that category in 2009.

The researchers have been tracking state proficiency standards since 2006. Their latest analysis focused on changes since 2009 when, like Tennessee, most states began adopting Common Core academic standards, then began retreating one by one from the nationally endorsed benchmarks.

Did the exodus from a consistent set of standards cause states to lower expectations for students? The researchers say no.

“Our research shows that most all the states have actually improved their standards, and Tennessee has probably improved the most because its standards were so low in the past,” said Paul Peterson, who co-authored the analysis with Daniel Hamlin.

The grades are based on the difference between the percentages of students deemed proficient on state tests and the National Assessment of Educational Progress, or NAEP, the exam administered by the U.S. Department of Education to measure what students know in math and English language arts. The narrower the proficiency gap between those tests, the higher the grade a state received.

Tennessee’s 2009 proficiency gap was 63 percent, an amount that Peterson called “ridiculous” and “the worst in the country” compared to 37 percent nationally.

In 2017, Tennessee’s gap narrowed to less than 3 percent, compared to 9 percent nationally, under revised standards that reached classrooms last fall after the state exited the Common Core brand.

“It’s a dramatic improvement,” Peterson said of Tennessee’s work to align its standards with national expectations.

Interestingly, in other states, the study found virtually no relationship between rising proficiency standards and test score growth — a finding that the researchers called “disheartening.”

“The one exception was Tennessee,” Peterson said of the state’s academic gains on NAEP since 2011. “It has not only raised its standards dramatically, it saw some student gains over the same period.”

Since 2010, higher academic standards has been an integral part of Tennessee’s long-term plan for improving public education. The other two components are an aligned state assessment and across-the-board accountability systems for students, teachers and schools, including a controversial policy to include student growth from standardized test scores in teacher evaluations.

Tennessee poured millions of federal dollars from its 2010 Race to the Top award into training teachers on its new standards. The process began in 2012 with large-scale Common Core trainings and shifted last year to regional trainings aimed at equipping local educators to prepare their peers back home for Tennessee’s revised standards.

PHOTO: Marta W. Aldrich
Karyn Bailey (left), a facilitator from Williamson County Schools, coaches elementary school teachers during a 2017 exercise on Tennessee’s revised standards for English language arts as part of a two-day training at La Vergne High School, one of 11 training sites across the state.

Implementation really matters. You can’t just make the shift on paper,” said Education Commissioner Candice McQueen, who will take part in a panel discussion on the study’s findings Tuesday in Washington, D.C. “You have to do the hard work to implement it on the ground. And that is a long game.”

The Harvard study comes on the heels of a separate but related report by pro-Common Core group Achieve that says Tennessee is essentially being more honest in how its students are doing academically. The state was called out in 2007 by the U.S. Chamber of Commerce because Tennessee tests showed students doing well, while national tests reported otherwise.

Both analyses come as Tennessee tries to regroup after a problem-plagued return to statewide online testing this spring.

While supporters of Tennessee’s current policy agenda fear that headaches with the state’s standardized test could undo the policies it may be getting right, Peterson said a study like Harvard’s can provide a birds-eye view.

“What happens over a period of years is a better way to look at how a state is doing,” he said, “because things can fluctuate from one year to the next.”

The Harvard research is funded by the Bill & Melinda Gates Foundation and the Walton Family Foundation. (Chalkbeat is a nonprofit news organization and also receives funding from both foundations. You can find the list of our supporters here and learn more about Chalkbeat here.)