National testing time

In first comparison with other large cities, Memphis students score poorly on national test

PHOTO: Laura Faith Kebede

Shelby County Schools has gotten its first marks on a national test considered the best way to compare student performance across locations — and the results aren’t good.

Compared to 27 large cities, the district landed in the bottom third of results from the National Assessment of Education Progress, known as NAEP. It also lagged far behind Tennessee’s scores — in which only about one-third of students met the basic standards for proficiency.

The results temper Tennessee officials’ frequently cited claim that the state’s students are improving academically faster than students in other states, but the state’s top education official said she was not surprised by Shelby County’s poor results.

The data was reported Tuesday from NAEP, a test given every two years that measures achievement among a sample of fourth- and eighth-graders across the country. Scores from 27 urban districts are reported separately. This is the first year Shelby County Schools has been compared with the other large urban districts.

NAEP is important because it has long been the only way to compare student performance across states, a distinction that has persisted even as many states adopted similar standards for what students should learn and begun testing students on whether they meet those standards. The exam, which is administered to a sample of students across each state and district, also serves as a check on state test scores.

The Memphis district ranked close to the lowest third in fourth-grade math and reading and in the lowest third in eighth-grade math and reading compared to other cities.

In addition, the scores from Shelby County Schools lagged far behind Tennessee’s scores. Candice McQueen, Tennessee’s top education official, said given Shelby County Schools’ recent performance on the state’s more rigorous test last year, the national city comparison was not surprising.

Superintendent Dorsey Hopson said the results tell him the district needs to stay the course in implementing a myriad of changes to better match instruction to state and national benchmarks.

“I think we’re doing the right work. I think it’s going to take time to implement all the things we want to implement,” he said Tuesday. “To me, it validates why you have to change. Expose kids and prepare kids for the rigor that NAEP measures as well as what the new Tennessee test requires.”

Hopson had said his district was driving the state’s gains over the past several years. He based his claims on the improvements students made on state tests that prior year. It’s still possible that Memphis students propelled the state’s outsized gains on NAEP between 2011 and 2015 — but the new data suggest that local students’ skills are hardly worth bragging about. Shelby County Schools, Tennessee’s largest district, makes up 11 percent of the state’s student population.

When the state introduced a new test in 2017, Shelby County’s performance dropped, along with the state’s. “Shelby County was in some of our lowest proficiency levels across the board, as many of our urban districts,” McQueen said. “So we need to continue to think about what are we doing specifically across all of our urban districts to provide support, resources, and intention to all students.”

The national test is low stakes for individual students and schools, but high stakes for politicians and education leaders who use the scores to tout or denounce education policies. The federal agency that administers the tests warns against connecting scores to specific policies.

A Shelby County Schools statement notes that Memphis fourth graders “surpassed their peers in cities like ours” in which 30 percent or more school-age children live in poverty.

“And with the exception of Atlanta, the cities our students are lagging behind have less than 30-percent of their school-aged children living in poverty, which makes the correlation clear.”

Of the participating school districts, Memphis’ 30 percent poverty rate for children was the fifth highest of the participating urban school districts, according to U.S. Census data. Their test scores mostly correlated with the percentage of families with children living in families, with the exception of eighth-grade reading, in which Memphis students scored higher than their poverty ranking.

Note: The score threshold for “basic” in the charts below signifies partial mastery according to NAEP’s standards.

Update, April 10, 2018: This story has been updated to include a statement from Shelby County Schools, comment from Dorsey Hopson, and more information about city NAEP scores and their poverty rates. 

We created several graphs and charts to examine NAEP scores in Memphis, Tennessee and across cities. Look through them all or skip to the ones you’re most interested in:
2017 NAEP Trial Urban District Assessment Rankings for 4th Grade Math
2017 NAEP Trial Urban District Assessment Rankings for 8th Grade Math
2017 NAEP Trial Urban District Assessment Rankings for 4th Grade Reading
2017 NAEP Trial Urban District Assessment Rankings for 8th Grade Reading
Comparing Memphis, Tennessee and nation: 4th grade reading
Comparing Memphis, Tennessee and nation: 4th grade math
Comparing Memphis, Tennessee and nation: 8th grade reading
Comparing Memphis, Tennessee and nation: 8th grade math
Large City NAEP scores by Poverty: 4th grade reading
Large City NAEP scores by Poverty: 4th grade math
Large City NAEP scores by Poverty: 8th grade reading
Large City NAEP scores by Poverty: 8th grade math

2017 NAEP Trial Urban District Assessment Rankings for 4th Grade Math

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Mathematics Assessments. / Graphic by Sam Park
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Mathematics Assessments. / Graphic by Sam Park

2017 NAEP Trial Urban District Assessment Rankings for 8th Grade Math

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Mathematics Assessments. / Graphic by Sam Park
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Mathematics Assessments. / Graphic by Sam Park

2017 NAEP Trial Urban District Assessment Rankings for 4th Grade Reading

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Reading Assessments. / Graphic by Sam Park
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Reading Assessments. / Graphic by Sam Park

2017 NAEP Trial Urban District Assessment Rankings for 8th Grade Reading

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Reading Assessments. / Graphic by Sam Park
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Reading Assessments. / Graphic by Sam Park

Comparing Memphis, Tennessee and nation: 4th grade reading

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1990–2017 Reading / Mathematics Assessments.

Comparing Memphis, Tennessee and nation: 4th grade math

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP).

Comparing Memphis, Tennessee and nation: 8th grade reading

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP).

Comparing Memphis, Tennessee and nation: 8th grade math

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP).

Large City NAEP scores by Poverty: 4th grade reading

Large City NAEP scores by Poverty: 4th grade math

Large City NAEP scores by Poverty: 8th grade reading

Large City NAEP scores by Poverty: 8th grade math

missed opportunities

A new report argues that students are suffering through bad teaching and simplistic classwork. Is that true?

PHOTO: Anthony Lanzilote

America’s public school classrooms are full of students who aren’t being challenged.

That’s the claim of a new report by TNTP, the nonprofit advocacy and consulting group, looking at student work and real-life teaching. Students are “planning their futures on the belief that doing well in school creates opportunities — that showing up, doing the work, and meeting their teachers’ expectations will prepare them for what’s next,” it says. “Unfortunately, it’s a myth.”

The study, called The Opportunity Myth, relies on TNTP’s exhaustive effort to get at what students are really doing in class by surveying them in real time, reviewing their work, and observing class instruction — a combination rarely seen in education research. Based on this data, the report argues that low-income students of color in particular are suffering through mediocre instruction and simplistic classwork while their teachers expect little of them.

“Students spent more than 500 hours per school year on assignments that weren’t appropriate for their grade and with instruction that didn’t ask enough of them,” the report says.

It’s not clear that the study’s methods can support such strong conclusions, though. TNTP’s claims turn on its own subjective way of rating instruction and assignments, and it’s unclear if different approaches would yield different results. And the paper examined just four districts and one charter school network, all anonymous.

That means the study is at once extensive and limited: extensive because it amounts to a massive undertaking to better understand students’ experience, but limited because it only examines a fraction of students in a fraction of classrooms in a handful of districts, none of which were chosen randomly.

Regardless of debates about the methods, the report may draw significant attention. The research of TNTP, previously known as the The New Teacher Project, has a track record of shaping policy, particularly with an influential 2009 report known as The Widget Effect, which focused on perceived flaws of teacher evaluation systems.

The latest study was funded by the Joyce Foundation, the Chan Zuckerberg Initiative, the Nellie Mae Education Foundation, the Walton Family Foundation, the Overdeck Family Foundation, and the Barr Foundation. (Chan Zuckerberg, Joyce, Overdeck, and Walton are also funders of Chalkbeat.)

In contrast with some of the group’s past work, the latest report concludes with few controversial policy recommendations, instead calling for higher expectations and a careful examination of disparities in school resources.

“We believe it’s time to move beyond important but narrow debates — from how to measure teacher performance to charter versus district to the role of standardized testing — and return to the basic guiding principle that brings us to this work: the right of every student to learn what they need to reach their goals,” the report says.

TNTP focuses on three large urban districts, one small rural district, and one charter network with three schools in separate cities. In all but the rural district, a majority of students are black or Latino, as well as low-income.

From there, TNTP got a handful of teachers from certain schools in each district to document their students’ work, collecting and photographing the assignments done by six students during three separate weeks. (Students had to receive parental consent and their names were removed from the work.) TNTP then assigned a rating to each significant piece of work, looking at whether it was on grade level, among other traits.

TNTP also had observers watch and then rate two lessons by each teacher using the group’s rubrics and surveyed teachers to determine their views on whether students could meet their state’s academic standards.

Finally, they surveyed students on their classroom experiences. TNTP used a novel approach for tracking student feelings, asking students whether they were bored or felt excited about learning at various points in a lesson.

In all, TNTP says, it reviewed nearly 1,000 lessons, 20,000 examples of student work, and 30,000 real-time student surveys. And the results, the report said, are grim: only 16 percent of lessons observed had “strong instruction,” and about a quarter of assignments were deemed “grade appropriate.”

This varied from district to district and classroom to classroom. In the most specific example provided in the report, one eighth-grade assignment asked a student to fill in the missing vowels from the word “habitat” after reading a short passage; in contrast, another required students to write a lengthy essay based on a memoir of one of the students to desegregate the all-white high school in Little Rock, Arkansas.

Student surveys were somewhat more positive: a narrow majority of students, about 55 percent, were generally engaged and interested during class, based on TNTP’s survey.

Students of color and low-income students tended to be in classes with worse instruction, fewer grade-level assignments, and lower expectations for meeting standards. That was correlated with slightly lower rates of test score growth.

All of that, TNTP concludes, amounts to a damning case against most of the classrooms in question and American schools in general. “Students spend most of their time in school without access to four key resources: grade-appropriate assignments, strong instruction, deep engagement, and teachers who hold high expectations,” the report says.

“The ‘achievement gap,’ then, isn’t inevitable. It’s baked into the system, resulting from the decisions adults make.”

But TNTP faces steep challenges in using its data to make such strong claims.

In addition to the districts, schools, teachers, and students not being chosen randomly, the report is not able to pin down whether those resources lead to higher achievement or definitively show why some students seem to have less access to the key resources it cites.

One of the report’s central claims, that increasing access to those resources will boost students’ academic performance, rests on relatively small correlations. In fact, the study showed little if any overall relationship between teachers’ observation scores and their effects on test scores.

“We need to be a little careful about asserting that by increasing one or more of the four resources we will necessarily improve outcomes for kids,” said Jim Wyckoff, a professor at the University of Virginia who sat on an advisory panel for the report, while also noting that he thought the basic theory of the report made sense.

The report’s appendix notes that “classrooms with initially higher performing students tended to get better assignments, better instruction, were more engaged, and had teachers with significantly higher expectations.” But other research has shown that observers tend to give unfairly high ratings to classrooms with more high-achieving students, meaning cause and effect could run the other way here.

TNTP’s measure of teacher expectations relies on teachers’ responses to statements like “My students need something different than what is outlined in the standards,” something that may be conflating high expectations with teachers’ views about the quality of their state standards.

Still, one of the main takeaways from the report — that low-income students of color have less access to good teaching — is generally backed by past research.

High-poverty schools have higher rates of teacher turnover and more inexperienced teachers, on average. Other research in a number of states and cities, including Washington state, North Carolina, New York City, and Los Angeles, has shown that teachers of low-income students are less effective at raising test scores.

Are Children Learning

Chicago is sending more high schoolers to college — but how to get them to graduate?

PHOTO: Adeshina Emmanuel / Chalkbeat
Mayor Rahm Emanuel, CPS CEO Janice Jackson, and other city officials convened at Michele Clark Magnet High School in the Austin neighborhood to announce the latest college enrollment statistics.

Senior Tanariya Thompson, 17, said she and her friends at Michele Clark Magnet High School are constantly asking each other about where they want to go to college. But they’re not just talking, they’re doing their research, too.

“In a lot of our seminar classes I see more kids on the computers applying for colleges instead of just sitting there looking or saying, ‘I ain’t going to college,’” she said. “We’re serious: We want to go to a college so we can become somebody. Next week, I will have my top three.”

Chicago Public Schools released data today showing that more students than ever before are enrolling in college. The mayor and district officials announced the encouraging figures on the West Side, at Michele Clark High School, where students said they’ve seen more energy, excitement and urgency among their peers around the idea of enrolling at college.

The data shows that 1,000 more Chicago Public School graduates from the Class of 2017 enrolled in college compared with 2016, a 4.8 percent increase and the biggest one-year jump in nearly a decade.

Chicago still has a problem with public school graduates staying in and completing college. In 2016, just 18 percent of ninth graders were projected to attain a bachelor’s degree within six years of high school graduation, and four-year college graduation rates have remained pretty stagnant since 2009, according to a fall 2017 report by the UChicago Consortium on School Research. (The report didn’t calculate two-year degree attainment).

But Mayor Rahm Emanuel called the latest enrollment data “an incredible statement about where Chicago Public School students are,” adding that nearly 90 percent of high school freshmen were on track for graduation.

“Every time they walk around and say, ‘not those kids, not from that school, not that background, not that ZIP code, not that family’ — you come here to Michele Clark and you tell these kids that,” Emanuel said, knocking on the wooden podium before him for emphasis.  “You guys have proved them wrong every step of the way.”

From 2010 to 2017, the college enrollment rate increased from 53.7 percent to 64.6 percent, according to the school district.  Officials credited everything from partnerships with OneGoal and other organizations focused on getting kids to and through college, to a summer text messaging campaign to nudge graduates toward completing action items along the enrollment path, and scholarships to city colleges for students who attain a B average or higher.

They also noted a shift in perspective.

“I think it’s because people have become more serious,” said Michele Clark Principal Charles Anderson. “I’ve seen it in action with people doing more college trips, people getting out to scholarship fairs, students having a different mindset.”

From 2016 to 2017, college enrollment rates for African-American and Latino students improved by 2.3 percentage points and 7.2 percentage points, respectively, according to the school district. The African-American college enrollment rate increased from 55.4 percent in 2016 to 57.7 in 2017, and the Hispanic college enrollment rate leaped from 59 percent in 2016 to 66.2 percent in 2017, according to district data.

Flanked by Chicago schools chief Janice Jackson and City Colleges Chancellor Juan Salgado, Emanuel said, “it used to be as a system, we were done just getting you to high school graduation, and our responsibility was over,” but now it’s different. The mayor added, “the biggest transformation is the mindset not just of our kids, but of the system.”

“It’s why we’re also making sure we set a goal that by 2019, every child has a plan for what comes next,” Emanuel said, alluding to a new CPS graduation requirement that demands every student “has a meaningful planning conversation with an adult, and graduates with a plan to map out their future.”

The data indicate more students are enrolling at City College of Chicago.

The district said 5.8 percent more students enrolled at city colleges in 2017 compared with the previous year. Of district graduates who attended two-year colleges in 2017, 84.5 percent enrolled at city colleges compared with 78.7 the previous year, according to the district. City Colleges Chancellor Juan Salgado praised the mayor and schools chief’s leadership, saying CPS’ gains were strong steps toward officials’ goals of “a more inclusive economy,” in Chicago.

“We also want to make sure that each of you has in a role in this economy, whether it’s downtown, or in our health-care centers, or at a logistics company, or engineering or manufacturing company or a tech company,” Salgado told the students. “This city will have a place for you.”

Officials said the climbing college enrollment rate mirrored the increasing number of district students earning high school diplomas, and also reflected district students’ overall strong academic progress. Yet the percent of students who enrolled in college in 2015 and were still enrolled the following year, 72.3 percent of graduates, is actually down slightly compared with 2010, when it was 72.8 percent.

That — and the low rates of Chicago Public School students who eventually graduate with a two- or four-year degree — are worrisome figures.

Furthermore, African-American and Latino students and students with disabilities still graduate from high school, enroll in and graduate from college at lower rates than the general population. It’s a sobering reminder of inequities in the school system.

Officials acknowledged that work remains to get more students to and through college.

That point that wasn’t lost on Michele Clark senior Naquanis Hughes, 17, who wants to study business in college but is still undecided on where. Hughes said staff, students, and even alumni offer this encouragement about getting through the hard knocks that some students encounter in higher education:

“If you come to a hard place, don’t just fall down, don’t just give up, keep pushing yourself.”