Datahead

Denver and Aurora schools showed modest gains on state tests. But gaps still remain.

PHOTO: Helen H. Richardson/Denver Post
Justin Machado, 9, reads on his iPad during his 3rd grade class at Ashley Elementary in 2015.

Two large Colorado school districts with substantial numbers of students living in poverty — Denver Public Schools and Aurora Public Schools — showed modest improvement in the second year of state tests measuring students’ mastery of tougher academic standards.

Despite those gains, both districts still lag behind the state average — and in Aurora’s case, the gap is substantial.

Overall, the state education department’s release Thursday of district- and school-level results from PARCC math and English tests amounted to a mixed bag for most, with scores creeping up in some subjects and grades, slipping in others or remaining steady.

The picture is further muddied by a lack of student growth data that measure changes in the same group of students over time — data state officials are still compiling. Low test participation rates in higher grades also call into question district and school level scores, officials concede.

The state last month released state-level 2016 PARCC results, which showed more elementary school students were meeting expectations, while middle school scores were flat.

For the first time since the state overhauled its annual testing system in 2014 to align to the politically controversial Common Core State Standards, teachers, parents and taxpayers are able to compare year-to-year results in most subjects and grades.

A student at Vista Peak in Aurora works on an assignment.
PHOTO: Nicholas Garcia
A student at Vista Peak in Aurora works on an assignment.

The results, along with other measures such as graduation rates, will determine the quality ratings of schools and districts. Those ratings will be released later this fall. For some schools, another round of poor results could mean state intervention — something that has never happened before.

Like last year, anti-testing sentiment ran highest in Boulder and wealthier suburban Denver enclaves, as well as in some rural districts. Older students were more likely to not take the tests while the overwhelming majority of younger students took them, also echoing last year’s trends.

At Boulder’s Fairview High School, just 72 of 514 ninth graders took the PARCC English test.

Joyce Zurkowski, the state’s director of assessments, said lower participation rates invite scrutiny.

“When I’m looking at a school with a high number of kids who met or exceeded expectations, with 98 percent participation, the confidence I can have in those results is higher than at a school with an even higher number of students who met or exceed expectations but had only 40 percent participation,” Zurkowski said.

The big five

Results from the state’s five largest school districts mostly mirrored statewide results, which most notably showed gains in elementary school math.

Denver Public Schools, the state’s largest school district, showed gains in all but one test. The district only lost ground in the percentage of students who met or exceeded the state’s benchmarks in seventh grade math, dropping by 2.6 percentage points. The district’s largest leap in the percentage of students who cleared the state’s benchmarks was in fourth grade English, with a jump of 5.5 percentage points.

“A decade ago, we were 25 points behind the rest of the state,” Superintendent Tom Boasberg said. “Now we’re about 3 or 4 points behind the rest of the state. Against that benchmark, we’ve made very consistent, very striking progress.”

Jeffco Public Schools, the state’s second largest school district, saw gains in math in every grade but sixth. But it lost ground across the board on the state’s English tests. Its largest drop, 6.4 percentage points, was in the ninth grade.

“We are pleased to be improving in math given the higher level expectations of the CMAS/PARCC assessments,” Superintendent Dan McMinimee said in a statement. “Reading will continue to be a focus for our district improvement planning and we won’t be satisfied until all of our students are meeting or exceeding state expectations.”

The state’s third largest school district, Douglas County, made gains in math at the elementary school level, but lost ground in middle and high school. The south-suburban school district had wild swings on the English test. Ninth graders gained 5.7 points on that test, but seventh graders lost 7 points.

Strong gains in math were made in Cherry Creek elementary schools. But sixth graders this year lost 4 percentage points. The state’s fourth largest school district had more mixed results on the English test. There was a 2.8 percentage point increase in the number of fourth graders who were at grade level in English. But there was a 2.3 percentage point drop at the eighth grade level.

Julie Skupa, Cherry Creek’s assistant superintendent of assessment and improvement, said the district could attribute its higher math scores in part to a new curriculum.

“It goes beyond knowledge and rote memorization, and requires students to do a lot of problem-solving,” she said.

Aurora Public Schools, the fifth largest school district, saw mostly improvements. Students made gains in every grade on the English test except for grades six and eight. Similarly, Aurora showed increases in the number of students who met state expectations on the math test in every grade except for sixth and seventh.

“We’re seeing the first overall increase in performance since the 2011 school year,” said Superintendent Rico Munn, who has been leading an aggressive school improvement agenda. “We hope we can attribute that to our increase in rigor and relevance, our effort to make sure we have a high-quality teaching staff, and focus on our strategic plan.”

Growing pains

Changes made to Colorado’s testing system during the 2015 legislative session — including who takes the tests and how they take the tests — complicated this year’s release. School leaders voiced their frustration over slow-to-be-released and incomplete data that in some cases can’t be used to make comparisons to last year’s results.

For starters, this year’s upper division math results can’t be compared to last year’s because different grade levels took those tests. In 2015, middle school and high school students were eligible to take the state’s most advanced math tests. However, after lawmakers eliminated testing in the 10th and 11th grades, only seventh through ninth graders were able to take those tests.

Seniors at Fairview High School in Boulder protested a standardized test in November 2014.
Seniors at Fairview High School in Boulder protested a standardized test in November 2014.

The upshot: you can’t compare results between the two years.

And for the second year in a row, Colorado has released results from its tests piecemeal. While schools are getting their results three months earlier than they did last year, the results are still slower than anyone expected.

“What they promised was that through the online system, we’d get results back sooner than later,” Skupa said, adding that it’s difficult for districts to make any meaningful changes after the school year has started. “It’s these bits and pieces that make it difficult to create a big picture view.”

Part of the slowdown this year, Zurkowski said, is that school districts were allowed to use pencil-and-paper tests on a much larger scale. But that’s only part of the problem, she said.

“I believe the PARCC consortium underestimated the complexity of scoring and reporting their assessments, especially in the first few years,” she said. “I do believe that not only Colorado but the PARCC consortium is committed to continue to find ways to improve that turnaround time.”

Paper-and-pencil tests are causing another set of concerns for the state. Last year, the state acknowledged that students who used paper tests performed better than they would have if they used online tests. Test that were impacted included the third grade English test and upper division high school math tests.

Zurkowski said that while the department has theories as to why the bump happened — students could have felt more comfortable writing out equations than keyboarding them — it doesn’t know for certain.

“There was a lot to sort through,” she said. “… Honestly, we don’t know what specifically the issue is.”

To ensure that didn’t happen this year, the state education department ran results from about 16 schools that used paper tests through a series of mathematical procedures ensuring students would get the same score whether they took the test on paper or online, Zurkowski said.

The additional steps make the results more reliable, Zurkowski said, but the department is still urging caution when it comes to looking at those schools.

“We’re not going to over-interpret at this point,” she said.

Search for your school

Use Chalkbeat’s database to search for your school’s individual results on the math and English tests. The green bar represents the number of students who met or exceeded the standards. The yellow bar represents the number of students who took the tests. State officials have cautioned that low participation rates could skew results.

Are Children Learning

Memphis schools in most need of growth see gains, but vast majority of students still not on grade level

PHOTO: Laura Faith Kebede
Principal Melody Smith discusses how students at A.B. Hill Elementary grew significantly in test scores.

Three years after one elementary school joined Shelby County Schools’ flagship school improvement program, Principal Melody Smith says growth is proof their efforts are working.

“We came together we battled, we cried, we fought tooth and nail, but in the end we kept our students in the center,” Smith told teachers as they reviewed the results a week before school began.

PHOTO: Laura Faith Kebede
Teachers at A.B. Hill Elementary discuss what makes an ideal school.

A.B. Hill Elementary School, which is part of the Innovation Zone, went from less than 5 percent of students reading on grade level last year to 15 percent in state test scores released Thursday. That jump earned the South Memphis school the state’s highest ranking in growth, but the scores also mean about 85 percent of students still don’t meet state requirements.

The iZone’s two dozen schools have been heralded for how much students have grown since 2012, especially when compared to the state-run Achievement School District, which heavily relies on private charter organizations to boost test scores, and scored the lowest in student growth.

But the challenge is far from over, and school leaders are looking for ways to improve faster.

State leaders generally look at three years of data before determining if academic strategies are working. And in the past three years, the state’s switch to online testing has been tumultuous, which has caused some district leaders and state lawmakers to question the results. But on national tests, Tennessee was held up as a model for student growth compared to surrounding states in a recent Stanford University study — even while the state is still in the bottom half of test scores nationwide.

PHOTO: Caroline Bauman
Antonio Burt became assistant superintendent in July over the Innovation Zone and other struggling schools within Shelby County Schools.

Only three schools in the iZone — Westhaven Elementary, Cherokee Elementary, and Ford Road Elementary — have more than 20 percent of students reading on grade level. By comparison, 16 schools surpassed that in science, five in math, and four in social studies.

“There was a lot of movement in our elementary schools,” said Antonio Burt, the district’s assistant superintendent for schools performing poorly on state tests. But “we’re going to need a laser light focus on our high schools and our middle schools.”

The district created the iZone to boost student achievement in schools performing the worst in the state, all of which are in impoverished neighborhoods. The state Legislature allowed principals to have much more autonomy on which certified teachers they could hire, pumped about $600,000 per school for teacher pay incentives, and added more resources to combat the effects of poverty in the classroom, such as clothes and food closets.

Now, entering its seventh year, the iZone is still outshining the state-run district, and students are still showing more growth compared to their peers across the state who also performed poorly last year. Nine schools in the iZone got the state’s highest ranking for growth, compared to just five last year when the state switched to a new test. (Scroll to the bottom of this story to compare test scores and growth for iZone schools.)

Of the 23 schools in the iZone last year, seven of them were high schools. None of the high schools had more than a third of students on grade level or above in any subject. Four of them — Raleigh Egypt, Melrose, Mitchell, and Hamilton — saw significant growth in at least one subject. Last year was Raleigh Egypt’s first year in the iZone under Shari Meeks, who previously was principal at Oakhaven Middle School.

PHOTO: Laura Faith Kebede
Clothes closet at A.B. Hill Elementary School in Memphis.

Burt said “the first big thing” that will be done to combat low reading scores in middle and high schools will be to strengthen curriculum. Adding curriculum for younger students played a part in boosting test scores that contributed to growth, leaders said.

Also, new reading specialists will teach a separate class for students who are the furthest behind on top of their normal English class. Before, teachers were responsible for catching up those students, or specialists would take them out of class to work on reading skills.

At the district level, Burt said science, social studies, math, and English advisors will be working more directly with teachers. And principal coaches will have more say in how and where those advisors concentrate their efforts.

Inside the school, Smith, the principal at A.B. Hill Elementary, said having teachers practice more difficult lessons in front of each other helped spur more ideas on how to make the curriculum work for their students.

Teachers said collaboration with others was key to figuring out the best way to improve test scores there. It was common for teachers to invite each other to sit in on lessons and give feedback.

“We would debrief with each other all the time,” said Brenda Pollard, who taught fourth-grade English and social studies. Now she says the foundation has been laid for higher achievement.

“It can be done,” she said. “We’re living proof it can be done.”

Below is a table of how iZone schools fared on state tests. Fields labeled “4.9” were hidden in state data, but are likely below 5 percent.

tar heel trivia

New education research? A good chance it’s from North Carolina.

PHOTO: Creative Commons/Boston Public Library

Barbeque. Basketball rivalries. The Blue Ridge Mountains.

Education research?

It’s something else North Carolina is known for, at least among a subset of social scientists.

“North Carolina has really done something special,” says Amy Ellen Schwartz, a professor and the editor of Education Finance and Policy, an academic journal.

“If you look over the last 20 years and focus on the highest quality work, it’s disproportionately work that comes from North Carolina data,” says Dan Goldhaber, an education professor at the University of Washington at Bothell.

North Carolina students aren’t more interesting or easier to find. But a disproportionate share of education research — and therefore, a disproportionate amount of what we know about how certain policies work — comes out of the Tar Heel State.

That’s because North Carolina has kept track of things like student test scores, teacher demographics, and school accountability data since the ‘90s, and also made that information more accessible to researchers than anywhere else.

It works well for those looking for data. But it also underscores a troubling reality: We know much less about how policies play out in places where data is hard to access — and in some cases, may be kept under lock and key for political reasons. That leaves the public to take the best lessons it can from a state that’s home to just 3 percent of the country’s public school students.

“The problem is that what you really want to do is look at lots of places,” said Schwartz, a professor at the Maxwell School at Syracuse University. “You want to be able to leverage the natural experiments and understand the variation in a way that’s really hard to do in one place.”

Of course, researchers in many cases do work productively with local officials to obtain data. And although it appears that North Carolina is the most commonly studied state in education policy, it is by no means the subject of the majority of academic papers. For instance, seven studies published in Education Finance and Policy over the last two years were focused on North Carolina — more than any other state or district, though over 30 others focused on K-12 schooling in the U.S used national data or data from elsewhere.

North Carolina’s popularity is tied to the fact that it is one of the few states where researchers can get student data (that has been anonymized) from a third party, in this case a research center established in 2000 that operates out of Duke University. In most states, the state education department or other state agency controls that information. Many states and districts lack the resources, streamlined systems, or staff capacity that North Carolina’s center has to meet researchers’ requests.

That center also separates policymakers and the keepers of the data — which may be crucial for ensuring information is made available.

“Not every place wants to open up their data and say, ‘Study what you want,’” said Schwartz. “The risk is that a researcher investigates something or casts it in a way that’s not positive for the school district.”

Goldhaber echoed this. “If you’re talking to somebody who’s involved with politics … they’re going to see everything through a political lens. And that when it comes to evaluating programs and policies, people often don’t see much upside,” he said.

In North Carolina, local researchers realized the importance of tracking students and schools over time, according to Duke’s Clara Muschkin, the faculty director of the data center.

When Goldhaber was studying schools there in the 1990s, he recalled, “There was a real belief that people ought to study these issues, and that was kind of pervasive under Gov. Jim Hunt.”

That extended to research that Hunt’s administration might not like. For instance, Goldhaber was interested in studying whether teachers who attained National Board certification were more effective in the classroom. Hunt was the founding board chair of the organization that awarded those certifications, and Goldhaber’s research had previously shown that certification types didn’t make much difference. But that didn’t stop the administration from providing that data to Goldhaber, who ultimately found North Carolina’s board certified teachers were particularly effective.

It’s impossible to say how often political concerns play a role in keeping data from researchers. When politics is involved, researchers themselves may not know, and if they do, they may not want to publicize it in hopes of eventually working out an agreement. (This reporter has heard frequent complaints about politics getting in the way of data access — but in most cases those are made off the record.)

A more subtle method of interference is when officials decide not to collect data in the first place that researchers might use to reach unflattering conclusions. California, Goldhaber said, is a particular culprit.

The largest state in the country has weakened, or declined to improve, its data systems since 2010, and the information that exists is not readily available to researchers. Governor Jerry Brown has argued that educational data is of little use to teachers and schools, and feeds into a test-focused mentality of schooling.

“You are not collecting data or devising standards for operating machines or establishing a credit score,” wrote Brown in a critique of the Obama administration’s Race to the Top program, which encouraged more data collection. “I sense a pervasive technocratic bias and an uncritical faith in the power of social science.”

Goldhaber has found it difficult to study the state’s education policies.

“There is just basic data that we could not get out of California,” he said, referring to a study he and colleagues are undertaking there.

Some places are becoming more cognizant of concerns about a lack of quality research about their schools. In Washington, D.C., the city council is considering funding an education research group and may make its data widely available to researchers. In California, some advocates and policymakers have pushed for improving its data systems, an idea the state’s likely next governor has backed.

In the meantime, those interested in key education questions — in California, DC, and elsewhere — can always look to North Carolina for answers. That’s largely a good thing, says Goldhaber.

“The fact that we are learning things in North Carolina is tremendously useful for informing policy and practice in other states,” he said.