analysis

Why Colorado’s testing opt-out movement could struggle to build on 2015’s big numbers

Fairview High School seniors protest CMAS tests during the 2014-15 school year (photo by Nicholas Garcia).

A year ago, thousands of Colorado students, parents and educators fed up with state tests used to hold schools, districts and teachers accountable said enough was enough.

More than 100,000 kids in grades three through 11 did not take new tests in math and language arts designed to measure how well they stack up to state academic standards.

Only about half of the state’s 11th graders took the exams. Just one grade — third — saw participation rates hit the 95 percent threshold set as the minimum required under federal law.

Colorado had become an epicenter of the opt-out movement.

This year, things are different.

Here are five reasons opt-out organizers may have a hard time building on that momentum this testing season:

There are fewer tests, and they’re shorter.

On the very last day of the 2015 legislative session, lawmakers passed compromise legislation that pared back state assessments. Not everyone was satisfied, but the political dynamics of shared governance meant pragmatism was going to win out.

The biggest headline was the elimination of PARCC language arts and math tests in 10th and 11th grades — and the preservation of ninth grade tests at the insistence of Democratic Gov. John Hickenlooper, whose veto pen held sway over the spring.

The biggest reason why PARCC opt-out numbers will plummet this year: High school sophomores and juniors accounted for more than half — 53,000 — of the roughly 104,000 Colorado students who did not take PARCC exams last spring, according to state data.

At the same time, PARCC exams got a little shorter for the states that remain in the increasingly skeletal multi-state collective that give the online Common Core tests.

At least one Colorado opt-out activist has said the goal for this year is to grow the number of opt-outs three-fold — to 300,000.

But former teacher Angela Engel, founder and executive director of United for Kids, a volunteer-run nonprofit that focuses not just on opt-out but more broadly on equity, innovation and organizing parents, cautioned against using raw numbers as a measuring stick.

With 10th and 11th grade PARCC testing gone, Engel argues the strength of the opt-out movement is better measured in the percentage of students that opt-out this spring.

“One of my concerns is that it’ll appear that opt-out is actually on the decline, when it’s not,” she said. “What is in decline is the amount of testing.”

Two opt-out hot spots last year report things haven’t changed much so far this year. In the Boulder Valley School District, about half of ninth-graders, a quarter of middle-schoolers and one in 10 elementary school students have opted out of PARCC this spring, officials say. The Cherry Creek School District says opt-out figures there are tracking with last year’s, too.

Colorado Education Commissioner Rich Crandall told Chalkbeat earlier this spring he expects much greater participation rates on this year’s state tests.

“I think we’re in a fair place,” Crandall said of Colorado’s testing landscape. “It’s definitely not too much testing, and I think the benefits outweigh the disadvantages.”

The political noise has quieted — for now.

A year ago, backlash against Common Core standards and the debut of PARCC tests dominated headlines. As online PARCC tests arrived — and not without glitches — state lawmakers got to work.

Not only was testing reform the signature education issue of the session, it was one of the biggest issues of the session, period.

Even before the 2016 General Assembly began, lawmakers signaled they were tired of it. Consider this, from a Democrat on the Senate Education Committee:

Sure enough, attempts to further chisel away at testing have fizzled this year.

On Tuesday, a bill that would have eliminated mandatory ninth grade PARCC tests died a swift death on the Senate floor. That the measure couldn’t even clear the Republican-controlled Senate — where it stood a far better chance of passage — tells you all you need to know.

There’s another reason for Colorado’s acceptance of the status quo this year: The long-in-the-making rewrite of the nation’s primary K-12 education law gives states greater flexibility to strike out on their own when it comes to what testing looks like and how it relates to accountability.

Crandall has not been shy about wanting to put Colorado at the front of the line in seeking this flexibility. Lawmakers have expressed interest in doing the same, when the time is right.

Colorado’s break from the testing wars may end up being a one-year cease fire.

There’s little evidence — so far — the opt-out movement is getting more diverse.

You might have seen the Twitter hashtag #optoutsowhite.

That the opt-out movement’s biggest numbers lie in wealthy white suburban enclaves such as south suburban Denver and Boulder County is indisputable.

Last year in Colorado, white students were disproportionately represented in the group that did not take PARCC tests, state data shows. Across all tested grades, about 78 percent of white students took the tests, while 85 percent of black students and 88 percent of Hispanic students did. Fewer than 9 percent of those who missed the exams last spring were eligible for free and reduced lunch status — an indicator of poverty.

It’s unclear how many students skipped the tests in protest or missed them for other reasons. (The figures above were calculated using the English tests; math participation was comparable).

An opt-out billboard near Abraham Lincoln High School in heavily Latino southwest Denver (Eric Gorski/Chalkbeat).
An opt-out billboard near Abraham Lincoln High School in heavily Latino southwest Denver (Eric Gorski/Chalkbeat).

As Politico noted earlier this spring, the opt-out movement is working to diversify it ranks and persuade minority parents that state tests are bad for their kids.

In Colorado, a small-scale billboard campaign encouraging opt-out this spring is focused not on minority communities but rather high-population centers, said Engel, of United for Kids. Five billboard designs picturing children who are opting out — one of them an African-American girl — are on display along the Front Range.

Jennilynne Coley is a Cherry Creek School District parent who is refusing to allow her African-American son to take state tests this spring at Laredo Middle School in Aurora.

Coley said she doesn’t think the tests accurately measure his intelligence or chances of success. She believes the tests are used unfairly against teachers in evaluations and determining their pay. And she is convinced the focus on testing unduly influences classroom teaching.

I asked Coley why more parents of students of color haven’t embraced the opt-out movement.

“When you don’t understand or know how the system works, you don’t understand you do have power and you do have a voice and you can speak up,” said Coley, who is self-employed.

She went on to say that minority parents are more likely to listen to school leaders and other authority figures, and are “busy taking care of their families and trying to survive.”

At the same time, standardized testing supporters are working to shore up support in minority communities, arguing that mass opt-outs will once again obscure achievement gaps that No Child Left Behind-era testing finally brought to the surface.

The opt-out movement — and the motivations behind it — is diffuse.

If it feels hard to get your hands around the opt-out movement, you’re not imagining things.

There is no one dominant organization, no one-stop shop for information. The movement is comprised of a few volunteer-run groups with not a lot of money, and engaged parents and students who are relying primarily on word of mouth and social media to spread their message.

“It’s pretty diffuse,” said former Jefferson County school board member Paula Noonan, who has an ear to the ground on opt-out doings. “It’s generally grassroots and under the radar.”

In late February in Los Angeles, I moderated an Education Writers Association panel on the opt-out movement. To open the discussion, I posed this question to Robert Schaeffer, a longtime critic of standardized testing who is public education director of FairTest, the National Center for Fair & Open Testing:

“What is the ultimate goal of opt-out? Is it to eliminate accountability-based standardized testing altogether? To reduce the volume of testing? To uncouple tests from school accountability and teacher evaluations? All of the above?”

Schaeffer’s reply, roughly paraphrased: All of the above.

While the loose structure and a wide range of agendas can prove beneficial, it also makes growing a movement challenging.

Opt-outs rates are likely to rise in some places — in ninth grade, certain high schools and some rural districts — but not enough to offset these other factors.

Let’s be clear: The opt-out movement in Colorado is not just limping along, nor is it going anywhere. It’s hard to imagine parents who opposed testing all of a sudden last year having a change of heart.

Schools in high-performing, affluent suburban areas with high opt-out rates last year may very well top them this year.

Carla Farris speaks about opting her daughter out of standardized testing in Douglas County last year (Photo by AAron Ontiveroz/The Denver Post).
Carla Farris speaks about opting her daughter out of standardized testing in Douglas County last year (Photo by AAron Ontiveroz/The Denver Post).

Parents in rural districts that didn’t see much opt-out activity last year may look at what happened in other rural communities last year and join the cause.

One thing seems a safe bet: lower participation in ninth-grade PARCC exams — possibly much lower.

It’s easy to envision high school freshmen looking at their older classmates and saying, “If they don’t have to take these tests, why should I?” An underwhelming number of students taking ninth grade tests would provide more ammunition to backers of finding alternative tests for freshmen.

Reports also are trickling in of poor participation on high school science tests — these are Colorado-only tests, not part of PARCC — in some school and districts.

Noonan, the former Jeffco board member, said Colorado opt-out activists are focusing efforts on elementary schools, where testing participation is high. You can see the logic — start ’em young. But it might be prove a tough sell.

“For the most part, at the elementary level, there is still value in knowing how our students are doing,” said Norm Alerta, director of assessment and evaluation in the Cherry Creek district.

Proponents of Colorado’s academic standards and aligned tests, meanwhile, hold out hope that year one was the toughest, and that more people will buy into the tests once they get used to them and get a fuller picture of student performance.

“Going into the second year, we’re going to start to see growth data and more information because we can compare the results to previous years,” said Reilly Pharo Carter, executive director of Climb Higher Colorado, which champions the tests.

The state’s official testing window closes Friday, and a full picture of participation rates will not be available for a few months.

Absent the political rancor, tests in two critical high school grades or significant progress in growing and diversifying their ranks, anti-testing activists’ goal of building on the momentum of last year’s big opt-out numbers faces a tough test.

failing grade

Why one Harvard professor calls American schools’ focus on testing a ‘charade’

PHOTO: Alan Petersime

Harvard professor Daniel Koretz is on a mission: to convince policymakers that standardized tests have been widely misused.

In his new book, “The Testing Charade,” Koretz argues that federal education policy over the last couple of decades — starting with No Child Left Behind, and continuing with the Obama administration’s push to evaluate teachers in part by test scores — has been a barely mitigated disaster.

The focus on testing in particular has hurt schools and students, Koretz argues. Meanwhile, Koretz says the tests are of little help for accurately identifying which schools are struggling because excessive test prep inflates students’ scores.

“Neither good intentions nor the value of well-used tests justifies continuing to ignore the absurdities and failures of the current system and the real harms it is causing,” Koretz writes in the book’s first chapter.

Daniel Koretz, Harvard Graduate School of Education

His skepticism will be welcome to families of students who have opted out of state tests across the country and others who have led a testing backlash in recent years. That sentiment helped shape the new federal education law, ESSA.

Koretz has another set of allies in some conservative charter and voucher advocates, including — to an extent — Secretary of Education Betsy DeVos, who criticized No Child Left Behind in a recent speech. “As states and districts scrambled to avoid the law’s sanctions and maintain their federal funding, some resorted to focusing specifically on math and reading at the expense of other subjects,” she said. “Others simply inflated scores or lowered standards.”

But national civil rights groups and some Democratic politicians have made a different case: That it’s the government’s responsibility to continue to use test scores to hold schools accountable for serving their students, especially students of color, poor students, and students with disabilities. (ESSA continues to require testing in grades three through eight and for states to identify their lowest performing schools, largely by using test scores.)

We talked to Koretz about his book and asked him to explain how he reached his conclusions and what to make of research that paints a more positive picture of tests and No Child Left Behind.

The interview has been edited for clarity and length.

Do you want to walk me through the central thesis of your book?

The reason I wrote the book is really the subtitle: we’re “pretending to make schools better.”

Most of the bad news that’s in this book is old news. We’ve been collecting evidence of various kinds about the impact of the very heavy handed, high-stakes testing that we use in this country for a long time. I lost patience with people pretending that these facts aren’t present. So I decided it would be worth writing a book that summarizes the evidence both good and bad about the effects of test-based accountability. When you do that, you end up with an awful lot on the bad side and not very much on the good side.

Can you talk about some of the bad effects?

There are a few that are particularly important. One is absolutely rampant bad test prep. It’s just everywhere. One of the consequences of that is that test scores are often very badly inflated.

There aren’t all that many studies of this because it’s not really a welcome suggestion. When you go to the superintendent and say, “Gee, I’d like to see whether your scores are inflated,” they rarely say, “Boy, we’ve been waiting for you to show up.” There aren’t that many studies, but they’re very consistent. The inflation that does show up is sometimes absolutely massive. Worse, there is growing evidence that that problem is more severe for disadvantaged kids, creating the illusion of improved equity.

Another is increasingly widespread cheating. We, of course, will never know just how widespread because there aren’t resources to examine the data from 13,000 school districts. Everyone knows about Atlanta, a few people know about El Paso, but that’s just the tip of the iceberg.

There’s obviously also — and perhaps this should be on the same par — enormous amounts of stress for teachers, for kids, and for parents. That’s the bad side.

I want to ask a little more about test score inflation. What is the strongest evidence for inflation? And let me give you two pieces that to me seem like potentially countervailing evidence. One piece is when I’m looking at research on school turnaround — like the most recent School Improvement Grant program and also turnaround efforts in New York City — these schools have been under intensive pressure to raise test scores. And yet their test scores gains on high-stakes tests have been pretty modest at best. The other example is the Smarter Balanced exam. The scores on the Smarter Balanced exam don’t seem to be going up. If anything, they’re going down.

The main issue is that score inflation doesn’t occur in the same amount everywhere. You’ve come up with two examples where there is apparently very little. There are other examples that are much worse than the aggregate data suggest.

In the case of Smarter Balanced, I would wait and see. Score inflation can only occur when people become sufficiently aware of predictable patterns in the test. You can’t game a test when you don’t know what irrelevant things are going to recur, and that just may take some time.

I’m wondering your take on why some of the strongest advocates for test-based accountability have been national civil rights groups.

One of the rationales for some of the most draconian test-based accountability programs we’ve had has been to improve equity. If you got back to the enactment of NCLB, you had [then-Massachusetts Sen.] Teddy Kennedy and [then-California Rep.] George Miller actively lobbying their colleagues in support of a Republican bill. George Miller summed that up in one sentence in a meeting I went to. He said, “It will shed some light in the corners.” He said that schools had been getting away with giving lousy services to disadvantaged kids by showing good performance among advantaged kids, and this would make it in theory impossible to do that.

Even going back before NCLB, I think that’s why there was so much support in the disability community for including disabled kids in test-based accountability in the 1990s — so they couldn’t be hidden away in the basement anymore. I think that’s absolutely laudable. It’s the thing I praise the most strongly about NCLB.

It just didn’t work. That’s really clear from the evidence.

I think the intention was laudable and I think the intention was why high-stakes testing has gotten so much support in the minority community, but it just has failed.

You mention in your book probably the most widely cited study on the achievement effects of No Child Left Behind, showing that there were big gains in fourth grade math and some gains in eighth grade math, but there wasn’t anything good or bad in reading.

Pretty much. There was a little bit of improvement in some years in reading but nothing to write home about.

So the math gains — and that was on the low-stakes federal NAEP test — they’re just not worth it in your view?

I think the gains are real. But there are some reasons not be terribly excited about these. One is that they don’t persist. They decline a little bit by eighth grade, they disappear by the time kids are out of high school. We don’t have good data about kids as they graduate from high school, but what we do have doesn’t show any improvement.

The biggest reason I’m not as excited as some people are about those gains is we’ve had evidence going back to the 1980s that one of the responses that teachers have had to test-based accountability is to take time out of untested subjects and to put it into math and reading. We don’t know how much of that gain in math is because people are teaching math better and how much is because kids aren’t learning about civics.

That’s, in my view, not enough to justify all of the stuff on the other side of the ledger.

When I’ve looked at some studies on the impact of NCLB on students’ social-emotional skills, the impact on teachers’ attitudes in the classrooms, and the impact on voluntary teacher turnover, they haven’t found any negative effects. They also haven’t found positive effects in most cases. But that would seem to at least in one sense undermine the argument that NCLB had big harmful effects on these other outcomes.

I haven’t seen those studies, but I don’t think what you describe does undermine it. What I would like to see is an analysis of long-term trends not just on teacher attrition but on teacher selection. A lot of what I have heard has really been, frankly, anecdotal. I was once a public school teacher and teaching now is utterly unlike what it was when I taught. It seems unlikely that that had no effect on who opts in and who opts out to be a teacher.

I don’t have evidence of this but I suspect that to some extent different types of people are selecting into teaching now than were teaching 30 years ago.

Can you talk about what you see as good versus bad test prep?

Something that Audrey Qualls at the University of Iowa said was, “A student has only mastered something if she can do it when confronted with unfamiliar particulars.”

Think about training pilots — you would never train pilots by putting them in a simulator and then always running exactly the same set of conditions because next time you were in the plane and the conditions were different you’d die. What you want to know is that the pilot has enough understanding and a good enough command of the physical motions and whatnot that he or she can respond to whatever happens to you while you’re up there. That’s not all that distant an analogy from testing.

Bad test prep is test prep that is designed to raise scores on the particular test rather than give kids the underlying knowledge and skills that the test is supposed to capture. It’s absolutely endemic. In fact, districts and states peddle this stuff themselves.

I take it it’s very hard to quantify this test prep phenomenon, though?

It is extremely hard, and there’s a big hole in the research in this area.

Let’s turn from a backward-looking to a forward-looking discussion. What is your take on ESSA? Do you think it’s a step in the right direction?

This may be a little bit simplistic, but I think of ESSA as giving states back a portion of the flexibility they had before No Child Left Behind. It doesn’t give them as much flexibility as they had in 2000.  

It has the potential to substantially reduce pressure, but it doesn’t seem to be changing the basic logic of the system, which is that the thing that will drive school improvement is pushing people to improve test scores. So I’m not optimistic.

One of things that I argue very strongly at the end of the book is that we need to look at a far broader range of, not just outcomes, but aspects of schooling to create an accountability system that will generate more of what we want. ESSA takes one tiny step in that direction: it says you have to have one measure beyond testing and graduation rates. But if you read the statute it almost doesn’t matter what that measure is. The one mandate is that it can’t count as much as test scores — that’s written in the statute. The notion that it means the same thing to monitor the quality of practice or to monitor attendance rates is just absurd

As I’m sure you know, research — including from some of your colleagues at Harvard — has shown that so-called “no-excuses” charter schools in places like Boston, Chicago, and New York City, have led to substantial test score gains and in some cases improvements in four-year college enrollment. Are you skeptical that those gains are the result of genuine learning?

It depends on which test you’re talking about. Some of the no-excuses charter schools drill kids on the state test, so I don’t trust the state test scores for some of those schools. I think it’s entirely plausible that some of those schools are going to affect long-term outcomes because they’re in some cases replacing a very disorderly environment with a very orderly one. In fact, I would say too orderly by quite a margin.

But those reforms are much bigger than just test-based accountability or just the control structure we call charters. It’s a whole host of different things that are going on: different disciplinary policies, different kinds of teacher selection, different kinds of behavioral requirements, all sorts of things.

A lot of the discussion around accountability, including in your book, is about the measures we should be using to identify schools. I’m interested in your take on what happens when a school is identified by whatever system — perhaps by the holistic system you described in the book — as low performing.

The first step is to figure out why is it bad. I would use scores as an opening to a better evaluation of schools. If scores on a good test are low, something is wrong, but we don’t know what. Before we intervene we ought to find out what’s wrong.

This is the Dutch model: school inspections are concentrated on schools that shows signs of having problems, because that’s where the payoff is. I would want to know what’s wrong and then you can design an alternative. In some cases, it may be the teaching staff is too weak. It may be in some cases the teaching staff needs supports they don’t have. It may be like in the case of Baltimore, they need to turn the heat on. Who knows? But I don’t think we can design sensible interventions until we know what the problems are.

charter talks

Hopson weighs charters as school turnaround tool for Shelby County Schools

PHOTO: TN.gov
Superintendent Dorsey Hopson leads Shelby County Schools in Memphis, home to Tennessee's highest concentration of low-performing schools.

Shelby County Schools Superintendent Dorsey Hopson has opened a crack in the door to charter school partnerships that might help his district avoid losing more schools to Tennessee’s turnaround district.

Hopson emailed his principals this week to clarify his recent comments to the editorial board of The Commercial Appeal about possibly recruiting charter organizations for turnaround work. The report’s original headline read: “Hopson says he’s willing to hand schools over to charters, if they have a plan for improvement.”

The superintendent quickly turned to Twitter to label the headline “misleading and inaccurate” and, as he sought to regain control of dialogue on the thorny matter, dispatched an email to his school principals.

“…It is my top priority to ensure all of our schools have the necessary resources to provide students with the high-quality education they deserve,” he wrote on Tuesday. “If the Tennessee Department of Education offers us the opportunity to select a charter operator that is willing to collaborate closely with District leaders to improve a school instead of losing it to the (Achievement School District), then I believe it is our responsibility to explore the option.”

Hopson’s comments hint at a potentially significant shift for a district that has battled openly with the charter sector over students being absorbed by the state’s 6-year-old turnaround initiative known as the ASD.

They also point to the tough spot that the superintendent is in.

On the one hand, the growth of the city’s charter turnaround sector has been a thorn in the side of local school leaders since 2012 when the state-run district began taking control of low-performing schools and assigning them to charter operators. Now with 29 Memphis schools, the ASD has siphoned off thousands of students and millions of dollars in an already under-enrolled and under-funded school environment — and made mostly anemic academic gains. (The local district also oversees about 50 charter schools that it’s authorized.)

On the other hand, Shelby County Schools has its hands full trying to improve a substantial number of struggling schools. It’s made some important headway through its Innovation Zone, which adds resources, extends the school day, and pays more to top principals and teachers who are willing to do some of the toughest education work in America. But the iZone is an expensive model, and few of its schools have exited the state’s priority school list.

In addition, some education reform advocates are lobbying to shift Memphis to a “portfolio model,” in which districts actively turn over schools to charter operators and manage them more like stocks in a portfolio. In other words, successful ones are expanded and failing ones are closed. Indianapolis has a robust portfolio model and, last fall, the philanthropic group known as the Memphis Education Fund took several Memphis school board members there for a tour. (The Memphis Education Fund receives support from several local philanthropies, including The Pyramid Peak Foundation and the Hyde Foundation. Chalkbeat also receives support from Hyde; read about our funding here.)

In his email to principals, Hopson said the school board ultimately would decide whether to authorize charter schools for the district’s turnaround work, and that he expects to discuss the matter with members in the coming weeks.

“All that said, I want to be very clear that my preference would always be to keep schools under the governance of (Shelby County Schools),” the superintendent added.

Hopson has been in discussions with the state Department of Education about several school improvement avenues available in Tennessee’s education plan under a new federal law. Among them is an option for Shelby County Schools to voluntarily convert priority schools to a charter, according to department spokeswoman Sara Gast.

One school board member told Chalkbeat he needs more information from the district and state before he would support any move forward. Chris Caldwell added that he thinks the board isn’t up to speed on options under the state’s new education plan.

“At this point, there’s so little information that I’ve been given,” Caldwell said. “I don’t want to conjecture what (a charter conversion) would actually will be like, but I have reservations with any kind of collaboration with the state.”

What would it take for such a shift to be successful?

One Memphis charter advocate says the ground rules are already in place because of a charter compact developed in recent years to address turf issues such as facilities, funding, and accountability.

“In order for a charter to manage a district school that’s underperforming and for it to be successful, that charter needs to have supports from the district to be successful,” said Luther Mercer, the Memphis advocacy director for the Tennessee Charter School Center.

The next school board work session is scheduled for Jan. 23.