Test talk

Want to boost test scores? Experts say Indiana must change teaching

PHOTO: Scott Elliott
An IPS School 79 student works on a test while her teacher looks on.

It’s been a common phenomenon in the testing world: A new exam is introduced, and the scores for it are significantly lower than for the old one — but then in the following years, come back up.

This “sawtooth” pattern has long been widely observed and written about, so it stands to reason that it’s also worked its way into the expectations of state education officials closely monitoring test results each year.

But in Indiana, ISTEP scores at the state level have been stagnant. In the two rounds of ISTEP tests since 2015, scores statewide have barely changed — 51.5 percent of students passed both English and math exams this year, compared to 53.5 percent in 2015. That year, with the switch to a tougher new exam, the number of students passing was 22 percentage points lower than on the previous test.

“The fact that (passing rates) are as flat as they are is striking,” said Derek Briggs, a test researcher and professor at the University of Colorado Boulder.

The suspicion that the flat scores are the new normal for Indiana — and that the lingering hope for big improvements in a short period of time could be unrealistic — has testing experts and educators speculating about why, and what could be done to nudge scores upward.

“The evidence that there’s a large test score gain after the first year or two, I think, is based on a different era of testing,” said Ed Roeber, a former Michigan testing director and consultant who has worked with Indiana on ISTEP.

Indiana is not alone in seeing flat scores. Michigan appears to be seeing a similar effect, with scores staying largely the same. But other states are still experiencing a sawtooth pattern.

Some experts suspect that the switch in 2015 to a tougher test based on new state standards has effectively ended the days of sawtooth results in Indiana.

Roeber said before No Child Left Behind, state tests and standards received far less fanfare from the public and even from rank-and-file educators. When a test changed, it might have caught schools off-guard and led to score drops. In subsequent years when students became more familiar with the format, scores would improve.

Now, it’s not just the format that’s changing — the questions themselves are more difficult and ask students to think in deeper ways than previously. More practice questions or better test-taking strategies aren’t a quick fix.

“People pay far more attention to the tests, so hard tests don’t get easier just by people becoming familiar with them,” Roeber said.

Briggs said the new test requires different skills that defy easy fixes.

“You could argue that if the standards were successful at focusing both instruction and redevelopment of assessments to that higher depth of knowledge, that these would be the kinds of things that are much harder to coach or teach the students how to beat on a test,” Briggs said.

If so, that puts added pressure on teachers and schools, experts said.

“The only way to really change student performance is to make sure they are learning what they are being taught,” Roeber said.

Roeber said teachers need to assess their own teaching and see how they can measure student learning every day, not just once a year. And schools need to give them the freedom, time and support to do it.

He calls these “formative tests” — different from the computerized Northwest Evaluation Association exams Indiana teachers have advocated for. Instead, he’s talking about asking students to do things like answer math questions on whiteboards after a lesson or having teachers randomly call on students to gauge widespread understanding.

At the school level, teachers should also meet within and across grade levels to ensure they are prepared for the students they get each year. They need to know students’ strengths and weaknesses and then adjust their instruction and curriculum accordingly.

“We spend about 95 percent of our resources telling people what kids do or do not know and less than 5 percent helping teachers assess their teaching,” Roeber said. “We keep emphasizing testing kids for how much they’ve learned and think that is going to change how teachers teach and what (kids learn), and it doesn’t.”

To be sure, there are many teachers and schools who already do this. Ayana Wilson-Coles, a second grade teacher at Eagle Creek Elementary School in Pike Township, is one of them.

Although Wilson-Coles now teaches a non-tested grade, she was previously a third-grade teacher. She’s still very involved in conversations happening between teachers, she said, and she knows what to look for in her own students to make sure they are prepared when they do get ready to test.

But the new ISTEP represented “a huge shift in thinking,” Wilson-Coles said. The standards now are “requiring kids to really think critically and have higher level thinking. I think sometimes teachers are not sure of how to do that … that has a lot to do with why we’re not seeing a change. That kind of thinking has to happen genuinely and you can’t force it.”

Wilson-Coles also thinks that the amount of testing being done is having a burnout effect. She remembers her third-graders being overwhelmed with tests — IREAD, then ISTEP, then the NWEA Map test.

“By the time they were doing the computerized tests, they were tired and done,” she said. “They do know how to think, but having them be motivated, and showing that” on still another test is a challenge, she said.

Bob Schaeffer, spokesman for The National Center for Fair and Open Testing, an organization that acts as a testing watchdog, said the flat scores could be indicative of a larger issue, but also show the accountability system as a whole isn’t leading to improvement — its stated purpose.

“It’s worth an investigation to try to see what’s going on and why things are flat,” Schaeffer said. “But (the state) should look at better ways to assess Indiana’s public school students that actually improves academic excellence and equity.”

Damian Betebenner, a consultant with the Center for Assessment who has worked with Indiana on A-F grades, said educators have to keep in mind that test scores are still an important indicator of “whether there’s problems or what types of problems you might have.

“It’s a number, and it’s a very valuable set of numbers, that can be used as part of a larger investigation as to what to do,” Betebenner said.

And, Briggs said, with one more year left of ISTEP, there should still be a way to draw some comparison between its scores and those on the next test, ILEARN, in 2019. It would depend heavily on the test vendor, content and structure, but “it’s not impossible that connections couldn’t be made to trend over time,” he said.

At the same time, Briggs said it may be too soon to fully explain the flat scores. More definitive research is necessary to establish any kind of trend, he said.

“At this point we should have evidence across the country,” Briggs said. “It’d be nice if we had that in a more systematic way.”

 

Literacy

It’s not impossible to teach teenagers to read. But it takes serious investment

PHOTO: Adeshina Emmanuel

Experts say it’s not impossible to teach older students how to read.

But late-stage intervention for students like Javion Grayer — a 16-year-old  who reads at a second-grade level after more than a decade in Chicago schools — takes daily practice and consistent one-to-one lessons with instructors trained to teach reading.

Such remediation, which expert say can’t happen in a general education setting or a large classroom, is something that most budget-strapped urban school districts, such as Chicago Public Schools, are ill-equipped to provide.

The district, though, insists it is taking steps to bolster literacy instruction. Just an hour after Chalkbeat published its profile of Javion — looking at how the teen fell so far behind and revealing the anguishing effects of his low literacy skills — Chicago Public Schools said it is developing a central reading curriculum that should be completed in the next two to three years. The goal: to ensure high-quality reading instruction and online library resources district-wide to support equitable access to content for readers at all grade levels, according to a district spokesperson.  

“It’s not acceptable for any student to leave our schools without being prepared for success, and the district will continue to build upon its academic improvements to ensure students have quality instruction and strong systems of support across the district,” said district spokesman Michael Passman in a statement. However, the statement skirted questions about specific interventions for older readers playing catch up.

What it will take to get students like Javion to grade level, is multipronged, literacy experts say.

“That’s obviously somebody who has fallen through the cracks,” said Rebecca Treiman, a professor of child developmental psychology at Washington University at St. Louis. “But there are ways to address these problems and it’s not like there’s a single age when somebody can read.”

Treiman, whose work focuses on spelling and literacy, echoed recommendations from other reading specialists, including nationally renowned literacy expert Louisa Moats, former Chicago schools reading director Tim Shanahan, and Alfred Tatum, dean of the college of education at the University of Illinois at Chicago — all of whom spoke to Chalkbeat.

After third grade, classroom instruction tends to move away from teaching students how to read and toward asking them to read in order to learn new material about other subjects.

For Javion and other older students with large literacy gaps, the experts recommended a return to basic phonics, in an effort to improve decoding ability, a daily diet of reading, and comprehension exercises. Shanahan and Treiman suggested a review of prefixes, suffixes, and common word roots. Moats prescribed helping students recognize commonly used “sight words,” and a focus on boosting vocabulary through reading and listening to texts. Treiman also recommended a curricular emphasis on students’ ability to perform everyday tasks, like filling out job applications and reading recipes. And Tatum was adamant about the need for culturally responsive curriculum, which takes into account students’ cultural identity, ethnic background and experiences.

However, even if such a rigorous remedial reading program were put in place in Chicago Public Schools, it’s still unclear how it would address the needs of older students. Such a program would also be optional for Chicago schools, since the district’s more than 640 schools, especially charter and contract schools, have a lot of autonomy to select curriculum. Since at least the early 2000s, Chicago has increasingly moved toward giving principals more freedom to choose what and how students are taught.

By contrast, the Houston Independent School District provides schools with guidance about the pace, scope, and sequence of English Language Arts instruction from pre-K-12, including “strategic reading and writing” curriculum for 9th and 10th graders who need remediation.

Having a centralized curriculum — while not a magic bullet —  is a way to ensure that students all start with certain building blocks of reading instruction, especially in the crucial early elementary years. And the earlier reading challenges are discovered, the better, experts say.

Reading was always painful for Javion Grayer, 16, but he wasn’t screened for special needs until seventh grade. Experts said he should have been evaluated early in elementary school.

Shanahan, formerly of Chicago Public Schools, recommended that the district push for about 50 minutes of phonics instruction a day in grades K-5.

“That’s how you figure out words in those early grades,” said Shanahan, a professor emeritus at the University of Illinois at Chicago, where he was founding director of the UIC Center for Literacy. “But I’d be very surprised if that’s true at more than half the [district] schools.”

Shanahan also served on the National Reading Panel, which Congress convened to evaluate research about teaching reading. The panel’s findings favored a focus on decoding words by breaking them into parts and sounding them out. That’s as opposed to the “whole language” approach many schools across the nation have pushed, where students learn to use pictures or context clues to fill in ideas and recognize words.

In 2017 the percent of students in Chicago performing at or above reading proficiency was 27 percent on the National Assessment of Educational Progress. That represents significant progress — in 2002, that number was 11 percent — but remains a cause for concern, given the lack of intensive reading instruction after third grade.

Students who fall behind after the third grade are more likely to be poor readers throughout life, and more likely to drop out of school, research shows. Students for whom English is a second language, especially recent arrivals to the United States or children whose parents lack English proficiency, are more prone to reading struggles. Meanwhile, serious gaps in reading ability often correlate with race and family income. Black and Latino students and those from low-income families tend to post lower test scores than their white and more affluent counterparts — largely the result of generations of racial and educational inequities.  

Moats said that such discrepancies often stem from “teacher training and the lack of it, the placement of less skilled, less experienced teachers in schools that are high minority populations or schools in less desirable neighborhoods.”

Reading failure, she said, “is way more common than anyone acknowledges. It affects way too many kids, and it’s unnecessary because it’s preventable; we know how to teach reading from decades of scientific work on how to teach kids to read.”

School discipline

Even as suspensions fall, Memphis students are being kicked out of school longer, data shows

PHOTO: Laura Faith Kebede/Chalkbeat
Memphis alternative school students work with local activist Keedran Franklin, in yellow, to brainstorm policy proposals to prevent other youth from being incarcerated. At the top of the list was mentoring and jobs. Just under that was a call to eliminate suspensions and expulsions and replace with fostering better relationships between teachers and students.

Hidden behind what Memphis education officials have said is good news when it comes to student discipline is a disturbing trend: As short-term suspensions have decreased, expulsions have increased.

Graphic by Samuel Park

Last year, Shelby County Schools handed down nearly 2,500 expulsions, according to district data. That’s about 300 more than in the 2015-16 school year — when the district already had one of the highest expulsion rates in the nation, according to federal data.

In one extreme example, a single high school issued one expulsion for every six students.

On average, expelled students were barred from school for 106 days, or more than half of the school year.

And while Tennessee law and district policies mandate expulsions for some offenses, 83 percent of the expulsions came at school leaders’ discretion. A third were for violations of relatively minor rules.

The expulsion data reveals mixed results for the district’s push to reduce discipline methods that keep students out of school. Shelby County Schools handed out 4,700 fewer suspensions last year than in the 2015-16 school year. Yet the rise in expulsions means that the total number of school days that students missed for discipline reasons actually increased.

Students spent about 14,200 more days in class because of the reduction in suspensions, based on the average three-day punishment. But the increase in expulsions resulted in close to 33,700 more missed school days.

The district’s black boys bore the brunt of the trend. They make up 38 percent of the district’s more than 100,000 students, but accounted for 67 percent of expulsions last year.

The data is raising questions among supporters of Shelby County’s discipline push, which launched as the federal education department pressed districts to limit suspensions and expulsions and reduce racial disparities among students who are punished.

“What we don’t want is for practices that we’re trying to replace to be replaced with practices that don’t support students,” said Cardell Orrin, executive director of Stand for Children, an advocacy group that has supported the district’s discipline efforts. “If we hide at all what are the real struggles, then we don’t identify the resources that are needed.”

(Tennessee defines suspensions as exclusions from school lasting less than 10 days; suspensions longer than 10 days are called expulsions. The district provided the length of expulsions only for students without disabilities, about 92 percent of expelled students.)

Graphic by Gabrielle LaMarr LeMee

District officials emphasized the reduction in suspensions and blamed the high expulsion numbers on charter schools and the state’s “zero-tolerance” law that requires expulsions for certain offenses. “Charters most often do not use in-school suspensions and progressive discipline, so their expulsions increase our numbers,” said a spokesperson, Natalia Powers.

But the district’s own data showed that charter schools, which have also worked to reduce suspensions, collectively reported 64 expulsions last year, 3 percent of the district’s total. And data the district provided showed that at most, only a quarter of expulsions were mandated by law.

District officials have also said they are confident that the district’s nine alternative schools for expelled students are serving those students well. One of those schools, G.W. Carver College and Career Academy, recently received state recognition for its work with expelled students and students who are transitioning out of incarceration. Students there meet with behavior specialists, mental health clinicians, and social workers, while families get support as well. District officials said as many as 40 percent of students choose to stay at Carver after their expulsion is over.

“They’re children and they sometimes make poor choices,” said Valerie Matthews, the district’s alternative schools director, at a recent conference for young men who attend alternative schools. “We keep them on track academically, we teach them how to modify their behavior, we work with them, we’re patient with them, we love on them, and it works.”

But students who are expelled are not required to enroll in alternative schools — something that the district’s school board has asked state legislators to change.

Matthews acknowledged that not all students who are expelled wind up in alternative schools. She said students who are excluded from school for less than a month frequently do not make the switch, and other students don’t attend because they cannot get to the alternative schools. The district provides bus passes, but the city’s struggling bus system can make using them challenging.

That reality means there are students who aren’t being educated because of their misbehavior — and, students say, could make them more likely to run into trouble in the future.

PHOTO: Laura Faith Kebede/Chalkbeat
John Chatman is a senior at G.W. Carver College and Career Academy, an alternative school in Memphis that recently received recognition from the state for its services for expelled students and those returning to school after incarceration.

“When they stay out of school, it’s not really a lesson learned, because the only thing they do is go home and chill, or go out and do the same stuff they been doing,” said John Chatman, a Carver Academy senior who was expelled from both East High School and Northeast Prep, another alternative school. “It takes away from education. It also puts them back into an environment that they were trying to escape from.”

Indeed, removing or excluding students from class does not address misbehavior, said Zoe Savitsky, an attorney who oversees education litigation and policy reform for the Southern Poverty Law Center.

“Would you ever say to a 6-year-old, ‘Get out of my classroom until you learn to read?’” she said. “You actually have to teach behavior skills you want them to have. And exclusionary discipline just ignores that reality.”

Principals in Memphis schools have a great deal of discretion in handing out discipline. Just 17 percent of expulsions in Shelby County Schools last year were required under Tennessee’s “zero tolerance” rules, which mandate expulsions for serious assaults on school employees; drug use or possession, and having a firearm at school.

Half of the expulsions were for what the district calls “other threats” and offenses that include fighting and assaults that do not result in serious injury.

And a full third of the expulsions were for what the district calls “rules violations” that could include skipping class or being out of uniform.

The district did not offer more detail about which rules being broken resulted in last year’s expulsions. But many of the behaviors that fall into that category are exactly the kinds of offenses that the district has targeted in its push to reduce suspensions.

2018 Youth Action Networking event

  • What: Students in BRIDGES’ advocacy program for formerly incarcerated youth will present their ideas on how to reduce both suspensions and expulsions to several district and county leaders. The event is sponsored by Bridge Builders USA, the University of Memphis Law Diversity & Inclusion Office, and Project MI.
  • When: 2 p.m. Saturday, Dec. 15
  • Where: BRIDGES, 477 N. 5th Street, Memphis, TN 38105

As part of that push, the district has hired more staff to dig into why students misbehave, crafted individual plans to help students improve, and rolled out alternative consequences before barring students from school. Now, 20 “behavior specialists” each work with about 10 schools to reduce suspensions, meaning that schools that don’t hire their own get only a little bit of support in working with students who misbehave.

“It kind of escalates, and [teachers] have to end up making an office referral for something that probably could have been redirected if they had the right tools,” Hargrave said. “If every school had someone who was an expert in trauma-informed practices or dealing with difficult behaviors along with the general staff, that would be ideal.”

Students suspended or expelled from school are more likely to have lower test scores, drop out of school, or become involved in crime than other students, links that led to the national push to reduce exclusionary discipline.

Advocates say that shift is especially necessary in Memphis, which has the highest rate in the nation of young adults who are not in school or working. Earlier this year, Orrin’s organization invited national expert Cami Anderson to train Memphis school leaders to prevent expulsions and suspensions and use alternative ways to discipline students.

Anderson previously was the superintendent of New Jersey’s largest school district and led New York City’s system of alternative schools for students with behavior issues. She said she’s not surprised expulsions went up while Shelby County Schools focused on reducing suspensions.

“If you only look at one, without intending, you can incentivize schools to take actions that have worse outcomes for kids,” Anderson said. “That’s true across the country.”