disappearing act

Data privacy worries shield thousands of Colorado test scores from public scrutiny

PHOTO: Nicholas Garcia/Chalkbeat
A HOPE Online student works during the day at an Aurora learning center.

The public will never know how Smoky Hill High School ninth-graders scored last spring on English tests that challenged them to do things like interpret ancient Greek poetry.

Nor will it know how many fifth-graders at Monterey Community School in Commerce City grasp concepts like identifying a story’s main idea. Or whether sixth-graders at Ortega Middle School in Alamosa can puzzle out the complexities of algebraic equations.

Those results from last spring’s PARCC tests were among roughly 4,000 data points shielded from public view — the result of a new, more restrictive state policy designed to protect individual students from being identified. More than 1 in 4 data points from the math and English tests are not available for public inspection because of the year-old policy.

The move to redact more data from the state’s publicly available standardized test results is a dramatic shift for a state known for rich and easily accessible educational statistics. Inspired in part by the State Board of Education’s zeal for student privacy, the change has sparked a new debate pitting data transparency advocates against student privacy supporters.

“It’s really problematic that we don’t know how thousands of kids at large high schools are doing,” said Lisa Berdie, policy director for A Plus Colorado, a school reform advocacy group. “(These results aren’t) just for punitive accountability decisions. It’s so communities and students and families have a sense of how their schools are serving them and whether they meet grade-level requirements.”

Officials at the Colorado Department of Education stress that districts and schools are receiving complete data sets, and that parents will be provided with comprehensive reports explaining how their students and schools are performing.

The new rules, state officials acknowledge, are among the most stringent in the nation.

Department of Education officials and State Board of Education members say that the state remains committed to using data for accountability, and that the rules are designed to make it impossible for a member of the public to pinpoint how a particular student performed on state tests.

“The intent and the purpose of the rules are important to protect individual privacy and prevent the identification of individual students through the manipulation of the data,” said Colorado Springs Republican Steve Durham, the state board’s chairman. “And as far as I’m concerned, it’s more important to protect those individual students than give the press something to write about.”

What do the new rules do?

Before 2015, when the new rules took effect, Colorado’s data rules were pretty simple. If fewer than 16 students at a school took any test — say, fourth-grade English — the state would not release the results.

The new rules say that if fewer than four students score at any one of the exam’s five proficiency levels, the state must redact results from that level and results from at least one other level. (If just the one were blacked out, doing simple math would allow someone to easily fill in the blanks).

Consider this example: Twenty fourth-graders at a school take a test and four place at Level 1, two place at Level 2, five place at Level 3, four place at Level 4 and five place at Level 5. The state would redact the results for Level 2 and one other level and report the rest.

In the last round of achievement results this month, the state didn’t release school-level results by individual proficiency level at all. Instead, it placed students in two broader categories — those who scored in levels 1, 2 and 3, and those who scored in 4 or 5, meaning they met or exceeded expectations.

Students at levels 4/5 were reported. If fewer than four students fell into that category, the scores were suppressed.

The state took an additional step that rubbed some schools the wrong way.

If a school’s results were withheld on any one test — and it is the only school in the district with redacted results on that test — the state took the additional step of also redacting the results on the same test from another school (the one with the fewest number of scores in the district).

The state believes this step is critical because it would be possible for someone to subtract the school’s student population from the district’s overall results to learn the school’s results.

That’s why Smoky Hill High School’s ninth-grade English scores were withheld this year. Because the results from the Cherry Creek School District’s alternative high school, Endeavor, were redacted, the state withheld the Smoky Hill scores because Smoky Hill had the fewest valid scores on the ninth-grade test.

Cherry Creek school officials are not happy about the new rules.

“A school that had more than 300 kids testing isn’t a school that should have any stars,” said Judy Skupa, the district’s assistant superintendent, referring to the typographical symbol the state uses when scores are redacted.

What’s the actual privacy threat?

According to federal law, the state must redact data that could allow any person using reasonable measures, such as basic subtraction, to figure out how a particular student performed on a test.

Joyce Zurkowski, the state’s chief assessment officer, explains it like this:

Imagine a neighborhood middle school had 100 sixth-graders take the state’s math test, and not a single one met the state’s expectations. Under the old rules, it would be very easy for folks on the block to know, at the least, the student next door did not pass the test.

That, Zurkowski said, would violate student privacy.

Here’s a slightly more complex example: Say a school had 17 boys and 10 girls take the third-grade math test. While the girls’ scores would be redacted under the old rules, their results could be determined through simple subtraction.

“You don’t get to know how your neighbor’s child performed,” she said. “We needed to do a better job of protecting that individual student’s data, that we hadn’t been doing historically.”

The Colorado Department of Education says it never received a complaint about privacy violations under the old, less restrictive system.

What’s the concern about transparency?

Advocates for more data are worried that the new rules will prohibit the public from knowing two things: which schools are doing poorly and which schools are doing exceptionally well, especially with traditionally underserved populations.

“That’s information we need to know,” said Luke Ragland, vice president of policy for Colorado Succeeds, a nonprofit group that advocates for school reform on behalf of the business community.

Advocates lay out another scenario: you could have a school where most students aced the exam, but that data could be withheld if too few students placed in the lower categories.

That’s what happened at West Ridge Academy in Greeley, as the Greeley Tribune reported.

“We’re not just hiding schools that are underperforming,” said Berdie, of A Plus Colorado. “We’re also hiding success stories.”

An even greater fear is that as the state breaks students into subgroups — students of color, students who qualify for free or reduced-priced lunches, students with special needs — the data will increasingly be redacted because not enough students are scoring in each category.

Starting in 2017, states will be required to break down student performance data into even more subgroups to include students of military families and those who are homeless.

Elena Diaz-Bilello, associate director of the Center for Assessment, Design, Research and Evaluation at the University of Colorado Boulder, said it will be increasingly difficult to draw any conclusions from state test score results if so much data is held back from the public.

“I don’t think the state thought through all the implications,” she said.

Where does the state go from here?

Many advocates in the education reform community are hoping the state softens the rules.

“My biggest fear would be that in the privacy environment we’re in right now, we’d swing so far in one direction and no longer have an opportunity to recalibrate,” said Dan Schaller, director of governmental affairs for the Colorado League of Charter Schools.

For the moment, the new rules only apply to state standardized tests given in grades three through nine, Zurkowski said. Other important measures that go into a school’s quality rating are not affected — including the SAT, graduation rates and growth data that show much students learn year to year.

The rules are not explicitly required by any law — not even Colorado’s landmark student privacy law passed earlier this year. Zurkowski said the policy is influenced by the federal Family Educational Rights and Privacy Act and guidance from the U.S. Department of Education.

That leaves room for the rules to shift, which Zurkowski said is a possibility.

“It’s an ongoing conversation,” Zurkowski said. “I”m not saying we hit this right. I know we haven’t hit this right. And these rules will continue to evolve. In the end, we got to balance transparency and privacy.”

focusing in

Black student excellence: Denver school board directs district to better serve black students

PHOTO: Helen H. Richardson/The Denver Post
Mary Getachew, 15, right, laughs with her peer mentor Sabrin Mohamed,18, left, at Denver's North High School in 2016.

Every Denver public school soon will be required to develop a plan to boost the success of black and African-American students by embracing their strengths rather than focusing on the challenges they face.

That’s according to a resolution unanimously passed Thursday night by the Denver school board. The resolution, which would also require district employees to take training on implicit bias, was shepherded by Jennifer Bacon, who was elected in 2017 to represent northeast Denver and is one of two black members on the diverse school board. Longer-serving board members said it was overdue.

“With good intentions, we were battling the idea that singling out a group of students was not acceptable,” said Happy Haynes, who has served on the board since 2011. “We were always talking about, ‘all students, all students.’”

In doing so, Haynes said, “we lost sight of so many of our students. So I really celebrate this change in our thinking.”

Denver Public Schools’ data show big disparities in how black students are served by the district. While 13 percent of the approximately 93,000 students are black,

  • 28 percent of out-of-school suspensions last year were given to black students,
  • 16.5 percent of students identified as having a disability were black,
  • Just 10 percent of students enrolled in rigorous high school courses were black.

The focus on black students comes after more than a year of relentless and high-profile advocacy from black parents and activists, and 2½ years after a damning report about how black teachers and students are treated in Denver Public Schools.

Known colloquially as the Bailey Report, it was based on interviews with black educators conducted by former school board member Sharon Bailey, who has studied racial dynamics in Denver. It found that black educators feel isolated and mistreated by the district, and perceive that black students are more harshly disciplined in part because the young white women who make up a sizeable portion of the teacher workforce are afraid of them.

The report led to a task force, which presented the district with 11 recommendations. Among them: offering signing bonuses to help attract more black teachers, making student discipline data count toward school ratings, and requiring each school to create a plan “designed to strengthen relationships between African-Americans and schools.”

Nearly two years later, none of that has happened. And much of what the district has done has been voluntary for teachers and schools. Meanwhile, the data keeps mounting.

Last year, 67 percent of black students graduated on time, meaning within four years of starting high school, compared with 78 percent of white students. On state math tests, 17 percent of black students in grades three through eight scored on grade level, compared with 65 percent of white students. The literacy gap was similar.

Avery Williams, a senior at George Washington High School, told the school board at a work session in December that “there’s an awkwardness around being black” in Denver schools.

“Teachers, specifically white teachers, don’t know how to act around me,” Williams said. Many of her classmates, she said, “do not know how to have respectful conversations because they’re afraid of being offensive or because they’re not educated in the right terminology.”

Michael Filmore, a junior at East High School, spoke about being one of only a few black students in his more rigorous classes, an experience Williams shares. After taking remedial classes his freshman year at East, Filmore said he decided to take all honors classes as a sophomore. He also took the public bus to school and was often late for first period.

“I would walk in the classroom and I would feel like I didn’t belong there,” Filmore told the board. “I felt uncomfortable and that I shouldn’t be in these classes. I was pressured. I eventually dropped the class. My junior year, I felt that I would never let myself down again.”

At that December session, Bacon expressed a desire to more explicitly address issues affecting black students. The district has put that kind of focus on students learning English as a second language, many of whom are Hispanic, after a federal judge found the district was violating their rights. Under that order, the district has developed specific methods for teaching English language learners. It requires all new teachers to get certified to teach them.

Bacon and others questioned why that hasn’t happened for black students, as well.

“It’s not because there’s a lack of effort, will, or love,” Bacon said in an interview. “I think it’s because we’re not organized properly and we don’t have an internal stake in the ground around expectations, outcomes, and accountability measures. People want to see DPS is doing that.”

Her fellow board members agreed. On Thursday, they took turns thanking her for bringing forth the resolution, which directs the district to do several things:

  • Require all schools, including district-run and charter schools, to review data about student academic performance, discipline, and referrals for special education to understand how each school’s black students are doing “on an individual level”
  • Require all schools to set goals for supporting black students that prioritize giving them “access to grade-level and more rigorous coursework”
  • Require school leaders to articulate how they will monitor progress toward their goals
  • Train all district staff on implicit bias and culturally responsive education
  • Conduct an “equity audit” to understand what the district is doing well and what it is not to figure out how it “can better prioritize the success of our black students”

It will now be up to new Superintendent Susana Cordova, who made equity a cornerstone of her bid for the district’s top job, to carry out the directive. The resolution gives her until May 31 to come up with a plan that would go into effect by the start of the next school year.

“We know that we have a painful and inequitable history of outcomes for our students,” Cordova said. “But facing this with courage, facing this in community, facing this with our stakeholders, our parents, our family members, our community members, and our students holding us accountable, I believe deeply in the ability of people to come together to solve these problems.”

making the rounds

Tennessee’s new education chief ‘very confident’ that online testing will be smooth in April

PHOTO: Shelby County Schools
Tennessee's new education commissioner Penny Schwinn (second from left) met with Douglass High School students and Shelby County Schools leaders Friday.

As Tennessee’s new education commissioner wrapped up her second week on the job by visiting four schools in Shelby County, Penny Schwinn said she feels “very confident” the state has learned from its mistakes in online testing.

During the more than three-hour ride to Memphis on Friday, Schwinn said she continued to pore over documents showing evidence that the corrections the state department staff have put in place will work.

“I feel very confident that our team has looked into that,” she told reporters in a press conference after meeting with students. “They’re working with the vendor to ensure that testing is as smooth as possible this year.” Currently the state is working with Questar, who administered TNReady online last year.

She also said the state’s request for proposals from testing vendors, which is already months behind, will be released in about two weeks.

PHOTO: Shelby County Schools
From left: John Bush, principal of Douglass High School; Penny Schwinn, Tennessee Education Commissioner; and Joris Ray, interim superintendent for Shelby County Schools.

“No later than that,” she said. “We hope and expect to have a vendor in place before the end of the fiscal year,” in late June.

The day Schwinn was hired, she said getting state testing right would be her first priority. Three years of major technical failures have severely damaged the trust educators and parents have in the state’s test, TNReady. It is the main measure of how schools and teachers are doing, but state lawmakers exempted districts from most testing consequences in 2018.


From Schwinn’s first day on the job: Tennessee’s new education chief wants to ‘listen and learn’ with school visits


Prior to talking with reporters, Schwinn said she heard “hard-hitting questions” from several students at Douglass High School in Memphis about what the state can do to improve education. Schwinn has said she will visit Tennessee schools throughout her tenure to ‘listen and learn’ by talking to students and educators.

Reporters were not allowed to attend the student discussion with Schwinn and some Shelby County Schools leaders.

Douglass High entered Shelby County Schools’ turnaround program, known as the iZone, in 2016 and saw high academic growth in its first year. But test scores fell this past year as the state wrestled with online malfunctions.

Timmy Becton Jr., a senior at Douglass High, said he hopes for fewer tests and more projects to demonstrate what a student has learned. Those kind of assessments, he said, can help a student connect what they are learning to their daily life.

PHOTO: Shelby County Schools
Tennessee’s new education commissioner met with students at Douglass High School and Shelby County Schools leaders.

“We figured it would be a different way to measure and see how much knowledge a student really has on a specific subject,” he told Chalkbeat after meeting with Schwinn during a student roundtable session. “It’s a good alternative to taking tests.”

He said he was “surprised and happy” to see Schwinn actively seek student perspectives.

“I really think that’s the most important part because students are the ones going to school every day,” Becton said. “So, if you want to find a good perspective on how to solve a problem, it’s really great to talk to the people who are actively involved in it and the people who are actually experiencing these problems directly.”

The state’s annual testing window runs from April 15 to May 3.