Data doubts

School discipline data mistakes have ripple effects for advocates battling the school-to-prison pipeline

PHOTO: ACLU of Southern California/Creative Commons

Faulty student discipline numbers for at least three Colorado school districts have muddied efforts to track suspension and expulsion trends, complicated advocates’ plans to press for change and caused state officials in one case to take the unusual step of correcting wrong data.

In early August, the advocacy group Padres & Jovenes Unidos, relying on numbers school districts provide to the state, reported surprising news: The rate of out-of-school suspensions in Colorado schools had shot up by 19 percent in the 2014-15 school year after years of decline.

However, three school districts portrayed as largely responsible for the spike all said their data were wrong, turning what had been a big jump in the suspension rate into a small one.

Getting school discipline data right is critical because advocacy groups campaigning to end the “school-to-prison pipeline” rely on the numbers to define the problem and push for changes in state law. Bad numbers can equal bad policy.

“We’re really, really invested in the accuracy of these numbers,” said Daniel Kim, director of youth organizing for Padres, which is widely known for its efforts to prevent harsh school discipline.

“These aren’t just numbers. These are students’ lives. These are the conditions they walk into every day at school. This is the environment of their school. That’s what these numbers reflect.”

Kim said the group plans to add a notation to its report, making clear that the data discrepancies could affect its findings.

The report attributed the 19 percent spike to big increases in the number of out-of-school suspensions in the Adams 12 Five Star, Colorado Springs 11 and Pueblo City 60 districts.

But shortly after the report’s release, Adams 12 came forward to dispute the findings. Officials said staff mistakes resulted in a report to the state that inflated its 2014-15 out-of-school suspension numbers by thousands and failed to include more than 100 expulsions.

After inquiries from Chalkbeat, Colorado Springs and Pueblo said their data was also wrong but in a different way. Out-of-school suspension numbers in both districts had been significantly under-reported to the state in 2013-14.

Taking into account numbers the districts say are correct, Chalkbeat calculated that the out-of-school suspension rate would have increased by just 2 percent in 2014-15. Similarly, the 7 percent suspension rate decline reported in Padres’ previous discipline report would have been around 1 percent had correct numbers been used for Colorado Springs and Pueblo.

National experts say inaccuracies in discipline data aren’t unique to Colorado and reflect growing pains in the push to elevate school discipline to the same high-profile status as academic achievement.

“This sloppiness wouldn’t be tolerated if it was test scores,” said Daniel Losen, director of the Center for Civil Rights Remedies at the University of California, Los Angeles.

“The more people start looking at school discipline…the more accurate the reporting will get.”

Already, he said, the spotlight on school climate and harsh discipline tactics is brighter because of new provisions in the recently reauthorized federal education law.

There’s also been a growing recognition that suspension and expulsion disproportionately affect students of color and can have devastating lifelong consequences—increasing the risk that kids will drop out and end up incarcerated.

Losen doesn’t blame advocacy groups like Padres for discipline data errors.

“It falls on the district,” he said. “This is the information age. There are so many ways in which we are good at tracking information about people and their experiences….It’s a little discouraging that schools are so slow in this regard.”

Genesis of a problem

Months before the Padres report came out, there were whispers among some of Colorado’s education insiders about sketchy discipline data.

The Colorado Children’s Campaign, spearheading an effort to reduce suspension and expulsion among young children, asked the state for data on suspension rates in kindergarten through third grade. The numbers that came back showed that Colorado’s public schools had suspended 7,433 students in those grades during 2014-15—with black and Latino children disproportionately affected.

When the Children’s Campaign circulated the aggregate data among key education groups, a few school district lobbyists and officials challenged its accuracy.

“We got a lot of, ‘Can this be right?’ sort of stuff,” said Bill Jaeger, vice president of early childhood initiatives and interim president at the Children’s Campaign.

Jaeger said that while discrepancies in one or two districts probably wouldn’t have affected the big-picture findings, concern about accuracy was one reason the group didn’t publish the numbers on its website or in its signature “Kids Count” report.

“The data question is something we want to get right absolutely,” Jaeger said.

The Children’s Campaign and other advocacy groups had hoped to see legislation on early childhood discipline during the 2016 session, but it never happened.

“There were a lot of folks who didn’t want to come to the table for a variety of reasons,” said Rep. Susan Lontine, D-Denver, who’s been actively involved in discussions on early childhood discipline.

“Some of it was they didn’t think the data was accurate, but there were other reasons, as well.”

Some of those reasons include disagreement about the best way to handle early childhood suspensions and expulsions in Colorado and the fact that preschool discipline data isn’t currently available from any state agency—leaving a big gap in what advocates know about the problem.

Why so much trouble with the data?

The school districts with problematic data cite various reasons, from self-inflicted “uploading errors” to state-level “coding changes.”

In Adams 12, officials submitted the incorrect data to the state in the summer of 2015. Eight months later—around the time the Children’s Campaign was passing around its K-3 data—officials there realized they’d made two major mistakes.

Not only had they reported nearly 9,000 out-of-school suspensions for 2014-15 instead of the actual 3,585, they’d reported expelling no students, when 121 had received that punishment.

They’d managed to look both better and worse than they actually were.

In an attempt to determine the extent of the data reporting problems, Chalkbeat contacted nine other districts, including four others singled out for having big increases in out-of-school suspensions, and the state’s five largest districts, four of which reported suspension declines.

Jeffco, Aurora, Douglas County, Cherry Creek, Greeley-Evans and Harrison said their numbers were correct. Denver didn’t confirm either way. Pueblo and Colorado Springs said they also spotted errors in the numbers that were sent to the state.

Those errors, along with the over-reporting error in Adams 12, made all three districts appear to have far larger year-over-year suspension increases than they really did.

Pueblo 60 spokesman Dalton Sprouse said administrators hadn’t gone back and looked at the erroneous discipline data until Chalkbeat asked about it. He said he’s not sure how the mistake, which resulted in a report of 1,161 suspensions instead of 2,264, occurred.

Colorado Springs 11 spokeswoman Devra Ashby attributed the mistake to a coding change at the state level. She said that in 2013-14, the district was asked to submit the total number of students suspended, not the total number of suspensions, which led to students with multiple suspensions only being counted once. As a result, the district reported 905 suspensions instead of more than 3,200. The next year, Ashby said, the district was asked to submit more detailed data.

But Jeremy Meyer, spokesman for the Colorado Department of Education, said the state has always asked districts to submit the same data year after year.

Of the three districts that cited problems with their discipline numbers, only Adams 12 asked the state to correct the data. This was in February, eight months after the data was first submitted.

The state initially refused, saying it was too late for corrections.

But last week, it reversed course. Meyer said fixes will be made to Adams 12’s numbers on the department’s website and reports submitted to the federal government.

Meyer said the decision represents a rare exception and doesn’t signify a change to the department’s policy not correct mistakes after districts officially submit their data.

That’s largely because the correction process is laborious—requiring a series of time-consuming steps at both the district and state level. Meyer said state officials decided to make the change for Adams 12 partly because the discrepancy was so large.

That doesn’t mean the changes will appear on every website or in every publication that published the original numbers.

“There’s a point when you release this information when it takes on a life of its own and it’s difficult to reel it back in,” Meyer said. “We don’t know who’s using it in what report.”

21st century schools

Five takeaways from a panel with author David Osborne, champion of giving schools autonomy and holding them accountable

Jennifer Holladay of Denver Public Schools, school board member Rosemary Rodriguez, Jeffco Superintendent Jason Glass and author David Osborne

When David Osborne was considering cities to spotlight in a book about reinventing American public schools, he started with the drastic overhaul of schools in Hurricane Katrina-ravaged New Orleans, moved on to Michelle Rhee’s controversial changes in Washington, D.C., and found his way to Denver.

Osborne, an author and consultant who specializes in documenting public sector reforms, is an advocate of charter schools and giving schools greater autonomy.

He was sold on Denver Public Schools, he told Chalkbeat, because of the district’s unusually long and strong track record of embracing charters — and by “performance improving going back a decade.”

DPS can point to successes: Enrollment and graduation rates are up, and state test scores are creeping closer to state averages. But the district also has yawning achievement gaps between minority students and white students, and between poor students and wealthier students, and teacher turnover is high.

Osborne was in Denver this week to discuss his book, “Reinventing Public Schools: Creating a 21st Century Education System,” and take part in a panel with Jeffco Public Schools Superintendent Jason Glass; Denver school board member Rosemary Rodriguez; and Jennifer Holladay, DPS’s executive director of portfolio management.

The Tuesday discussion at the University of Colorado Denver School of Public Affairs was hosted by A-Plus Colorado, Democrats for Education Reform, the Donnell-Kay Foundation, the Gates Family Foundation, the Progressive Policy Institute (where Osborne works) and The 74. (The Donnell-Kay Foundation and the Gates Family Foundation are funders of Chalkbeat).

Here are five themes that emerged from Osborne’s remarks and the panel discussion that followed:

Autonomy, accountability, choice … and districts getting out of the business of operating schools altogether

In his book and his Denver talk, Osborne laid out the ingredients he believes will improve America’s public school system. It starts with autonomy: giving school leaders the freedom to do whatever it takes to help kids. This is why Osborne is a steadfast believer in charter schools, which are operated independently and can make their own decisions about school calendars, hiring and firing, and curriculum, among other things. Next is accountability: schools that succeed grow and replicate, and those that fail are closed. Then you give parents a choice among schools. Finally, the most politically difficult piece: Osborne thinks school districts should get out of the school operating business altogether. All schools would be independently operated, with districts doing the “steering” and school operators doing the “rowing,” he said. “Denver has done a pretty good job of doing both, but it’s really unusual,” Osborne said. Not everyone is pleased with how DPS is doing both. As Chalkbeat reported last spring, charter school operators complained that they were not getting a fair shot in the competition to replace two schools being closed for poor performance.

Denver’s “family” of schools is sometimes in need of counseling

Denver is known nationally for its “portfolio” system that includes traditional district-run schools, charter schools and innovation schools, which enjoy many of the same freedoms as charters. Holladay, who oversees that portfolio, has a different term for it. “I sometimes think of it more like a family of schools, because we fight a lot,” she said. “… There are periods where the family of schools has to go to marital counseling, and there are periods where we do extraordinary work for children with each other.” Holladay suggested that collaborative work could be improved and expanded. “If you look at the portfolio of schools, the ones that struggle the most are single-site charters because they operate in isolation,” she said. Holladay cited new models of collaboration popping up around the country, including innovation network schools in Memphis and Indianapolis, as well as Denver’s own Luminary Learning Network. That network of four schools is part of an innovation zone, which gives schools even more autonomy than regular innovation schools. So far, the effort has shown mixed results, with two of the four schools posting low growth and slipping scores on the latest state tests.

Indianapolis could prove to be a model for Denver and other cities

Osborne had plenty of praise for Indianapolis, the only city in the country where the mayor authorizes charter schools. Much like in Denver earlier, education reform advocates in Indianapolis poured money and energy into winning control of the school board. When that paid off, the board coaxed the superintendent into retirement, brought in a new leader, and the district successfully lobbied the state legislature to pass what’s called the Innovation Network Schools Act. In Colorado, innovation schools enjoy many of the freedoms afforded charters, but they’re still run by school districts. Not so in Indiana, where nonprofit organizations and outside charter operators operate innovation schools that are still considered to be district schools. To Osborne, a true believer in as much autonomy as possible, this is the way to go. He said both Denver and Washington, D.C., should look to the Hoosier state for inspiration. On the most recent Indiana state tests, innovation schools’ scores were still pretty dismal, but showed the most improvement compared to other school models (schools taken over by the state fared the worst). Note: Critics say innovation schools had an easy road to higher grades under Indiana’s system. “Within the district framework,” Osborne said, “it gives you the dynamics that can lead to higher performance.”

Segregation and quality school authorization are challenges to choice

Glass, the Jeffco Public Schools superintendent, clearly was meant to be a contrast to Osborne and the DPS folks who are all-in on charter schools and autonomy with accountability. On the job since July, Glass is a sort of reformed reformer. He’s had a change of heart on strategies such as tying teacher pay to student performance (used to champion it, is now against it) and he came strongly endorsed by the Jeffco teachers union. Glass on Tuesday praised Denver for its nationally recognized centralized admissions system and for taking “courageous stands” on school authorization and closing schools that need closing. He also took issue with aspects of Osborne’s book. Glass said Osborne overlooked “some of the segregation issues that come with school choice — that are a problem every place you implement school choice.” Osborne’s book “also glosses over some of the difficulty” of being a quality school authorizer, Glass said. Charters often “come with low quality applications, and if they keep fighting, they get the state board or some other entity to roll over and approve them,” he said. “That’s a problem.” (Glass is not alone in this sentiment). Osborne conceded that some charter school authorizers are “awful” and advocated that less is more: one strong authorizer in each city and an “escape valve” at the state level if charter applicants are “somehow treated unfairly.” That’s essentially the Colorado model.

Denver is not getting out of the business of running schools any time soon — maybe ever — but is likely to loosen the reins

DPS has shown no indication of backing away from operating schools altogether. If anything, it has invested greater energy into replicating promising district-run models, including opening spinoffs of Grant Beacon Middle School and McAuliffe International School. The district in 2018-19 is planning on giving all innovation schools more freedom over their budgets, allowing them to opt out of district services (like, say, help from the district’s family engagement office), and spend that money on something else. Osborne’s advice to DPS: “Be bold. You have gone partway down this path, but there are other steps you can take.” Among his suggestions: giving schools even more autonomy, recruiting charter networks from outside the city, adding charter preschools (which are already here) and adult schools, and improving its school rating system, known as the School Performance Framework. That system, which is adjusted and tweaked frequently, is a common target of criticism, in part because it gives much more weight to how much students are improving than it does to whether they are proficient in a subject.

Future of Schools

Cary Kennedy, a Colorado gubernatorial candidate, wants to give teachers a raise. Here’s how.

Cary Kennedy (Photo by RJ Sangosti/The Denver Post)

Former state treasurer Cary Kennedy wants to give Colorado teachers a sizeable bump in pay.

One of several prominent Democrats running for governor, Kennedy, who helped write a constitutional amendment to increase school funding, released her education plan Thursday. The main goal is to get every Colorado kid into college or the work force by the age of 19. To do that, she’s putting her political capital into making the state’s teachers happier.

The proposal calls for more pay, a scholarship program to attract more teachers of color, and giving teachers a larger say in the state’s testing and accountability systems. She’s also calling for school districts to adopt a school improvement policy favored by teachers unions that calls for more welfare programs in the schools to combat the effects of poverty.

In a conversation with Chalkbeat, Kennedy discussed how she plans to reform the Taxpayer Bill of Rights to send more money to school districts, how she was influenced by attending a historically integrated high school in Denver and why access to free preschool is important.

This interview has been edited for length and clarity.

You’re calling for teacher raises. How can a governor in a local control state such as Colorado where salaries are set by school boards do that?

Provide the funding. I’m not proposing that I dictate to school districts what they pay their teachers. I’m proposing that the state provide adequate resources to school districts so that they can adequately pay our teachers.

We read every day about the teacher shortage in Colorado: 3,000 teaching positions right now that are not being filled. And it’s in large part because teachers can’t afford to work here. They can’t afford to live here. We pay our teachers among the lowest salaries in the country. I hear from principals that they are losing their best teachers because they’ll make $20,000 or $30,000 more for the same job if they go teach in another state.

This issue is impacting rural Colorado the most. We have 90 school districts where the average teacher salary is below $40,000. We can’t compete for the talent pipeline. And we’re not giving our great teachers who are doing amazing work in classrooms every single day the support they need and deserve, the professional pay they need and deserve.

This all goes back to what do we want to accomplish in education. And I’ve laid out the goal that every single student in Colorado, by the age of 19, is ready for higher education, has an employable skill, or both. And it takes great teachers. We know from data that the most important thing for a student’s success is the quality of the teachers that they have. We want Colorado to be the best place in the country to teach.

Have you put a price tag to this?

My goal as governor is to bring Colorado teacher salaries at least up to the national average and to eliminate what we call the teacher pay penalty, which is the difference between what a teacher gets paid and what someone with a comparable level of education earns in other professions. We want to eliminate the disincentive to teaching. Bringing Colorado up to the national average, we estimate to be around $240 million a year. To eliminate that teacher pay penalty is around $500 million.

It does not make any sense that Colorado’s economy ranks No. 1 in the country right now, according to U.S. News and World Report, and our investment in education ranks at the bottom. We’re living the consequences every day by having the teachers leave the profession and by having people who want to teach say they can’t do it in Colorado.

Where are you going to find that kind of money?

We need to recover what TABOR has taken away. TABOR (the Taxpayer Bill of Rights which limits how much money the state can collect from taxpayers) has put us in a hole. As our economy has grown over the last two decades, our schools have not benefited from that economic growth. TABOR has prevented us from doing that. We’ve been cutting school budgets for 25 years. That isn’t what anyone in the state wants. They want Colorado — we want Colorado — to have the best education system in the country.

I will lead, as governor, to build the coalitions to get back what TABOR has taken out of our schools.

Conventional wisdom would say this promise is a huge political risk.
I’ve called for permanent TABOR-reform for my entire career. I have helped lead our state to have responsible fiscal policy, a balanced budget throughout the economic crisis, low to moderate debt levels. We pay our bills; we keep our taxes in Colorado low. All of that helps us remain competitive and attract capital investment in our state.

But we can not continue the prosperity we are enjoying today if our kids growing up can’t compete for the jobs we’re bringing here. We have to give our kids the educational foundation they need to be competitive for those jobs.

And it’s also how we’ll make sure our prosperity reaches everyone. Right now we have people who are being left out, who are being left behind. And a great public education system is the only way we’ll ensure our progress reaches everyone.

What does TABOR reform look like? How do you want to change it?

It would be to allow our state to keep up with growth. TABOR has said as your economy grows, you are not able to generate taxes off that growth to invest in your infrastructure or education system or your health care system.

People in Colorado know they’re sitting in traffic. Our streets are crowded, our schools are crowded, we’re underinvested in education. That’s because we have not been able to keep up with the demands of a growing economy. We can keep low taxes, we can keep the protections for taxpayers that are in TABOR. There is bipartisan support today to modify the caps in TABOR to keep up with growth. You will see me lead on that as governor.

There are folks out there who say public schools receive the largest chunk of the state’s budget and don’t need more money. It’s a question of them spending the money in a more efficient way. What do you say to those folks?
We’ve been cutting school budgets in Colorado for three decades. Half of our school districts in Colorado today have had to cut back to a four-day school week. We pay our teachers among the lowest salaries in the country. We’ve got 90 school districts with an average salary below $40,000. We have to make the investment to compete as a state for the kind of economic progress that we all want in Colorado. People in Colorado know that education needs to be our priority and we’re not where we need to be as a state.

You want to expand the role teachers play in assessments, teacher evaluations and school quality ratings. What does would that look like in practice?

We know from (the state’s teacher) survey that teachers don’t believe the current assessment data is helpful to them in their efforts to improve student learning or improve their instructional strategies. We want this assessment data to help support our teachers in really knowing and understanding how their kids learn and what their students need. Teachers need to be much more involved in developing that process.

To do that would cause a lot of upheaval in the current system. One of the things we’ve heard is that teachers and principals are tired of change. How do you balance rethinking those systems while making sure there isn’t another upheaval and more ‘unfunded mandates?’

I think teachers and principals right now feel they’re spending a lot of time on an assessment system that isn’t giving them meaningful information and they’d welcome the opportunity to spend the time to bring their voice to the table on how we can do a better job in this state.

Your proposal is heavily focused on teachers. I can imagine there is someone out there thinking you’re just angling for the endorsement of the teachers union.

I’ve spent my career trying to improve our public schools. I graduated from Manual High School. I’ve seen the challenges in our public schools firsthand. I had great teachers. I’ve always done the work I’ve done because I really believe that public schools are where kids get opportunities to go on and lead productive lives. I grew up with three brothers and sisters who joined my family through the foster care program. I also have a sister who joined my family through her church. So, I’ve grown up with brothers and sisters who didn’t have the opportunities that I had been given. And I saw firsthand how important the opportunities are that kids received through their public schools to determine their future success.

You’re the product of an integrated school. Integration has become a hot topic in education circles again. How have you been thinking about integration?

The achievement gap in Colorado today, the difference between how white kids are performing compared to students of color, is unacceptable. We have the second-largest achievement gap in the country. There are too many students of color in Colorado who are being left behind, who aren’t getting the opportunities they deserve. That’s why this proposal is so important. We want all of our kids regardless of where they grow up, regardless of their family income, regardless of their background, to be successful in school.

We want our schools to reflect the diversity and the richness of our communities. That will happen if all schools have and provide meaningful learning opportunities with high-quality teachers.

You see in my proposal a real focus on attracting and retaining teachers who also reflect their student makeup: Latino teachers, black teachers, teachers of all socioeconomic and racial backgrounds so kids going to school have role models they can look to.

We want integrated, inclusive community schools that reflect the diversity of the state’s population.

You’re calling for universal access to preschool. So is your opponent Rep. Jared Polis. In fact, it’s his central education campaign promise. How are your proposals different?

I worked for Educare Colorado and developed the school readiness legislation that is current law that expands opportunities for low-income kids to attend high-quality preschools. I was also involved in the Colorado Preschool Program, the Denver Preschool Program. Even with these successful efforts, only half the kids in our state attend preschool and full-day kindergarten. Half the kids in our state start behind. And teachers will tell you that it’s really hard to catch them up. They stay behind and they finish behind. As a state it’s imperative that we make sure all kids have access to high-quality early education.

It’s going to be more money.

It’s an investment that’s critically important. We can look to private and public partners. But it is not acceptable that we prevent 4-year-olds from attending a high-quality early learning opportunity.

Is this another ballot question or is this something you can do with existing revenue?

You have to prioritize it. It’s building a statewide vision for what our public education system in Colorado can and should be.

You call for expanding so-called community schools. That’s an amorphous term that means something different depending on who you talk to. What does a community school mean to you?

Community schools are focused on engaging the community in supporting the school. It’s bringing in two-generational learning so that parents and students can learn together and parents can support their student’s learning. But it’s also bringing in wrap-around services. A lot of kids arrive at school with challenges. It’s social-emotional, growing up in stressful home environments, suffering from toxic stress. We have kids who are coming to school who are homeless and who need additional support in their learning. If we want all kids to be successful, we have to address the challenges kids show up every day with.