testing testing

Summer of testing debate launches with release of study

Updated May 14, 4 p.m. – School leaders, teachers and parents are worried about the time demands that new state tests will impose and the impact on classroom learning, school schedules and staff time, according to research done this spring for the Colorado Department of Education.

The study, done by the California-based education-consulting group WestEd, included surveys of district assessment coordinators, focus groups in selected districts and additional focus groups with parents and teachers.

“The number one thing we heard over and over from focus groups was the impact on instruction,” said Sheila Arredondo, WestEd lead researcher, told the State Board of Education on Wednesday.

That work was done from February through April, before new online science and social tests were given to more than 250,000 students in four grades and before “field tests” of the coming CMAS assessments were administered to more than 12,000 students in 85 districts. (Details of how that went were presented to the State Board of Education on Wednesday; read the story here.)

WestEd researchers are beginning new surveys and focus groups this month to gauge attitudes now that those tests actually have been given.

Arredondo said she’s already talked to some districts about their experience with spring testing. “The kids seemed to handle it beautifully … it went pretty well.” But she said administrators remain worried about being able to handle testing next year when CMAS exams have to be given in every grade starting with the third.

“I think the study was more validating than surprising,” said Jill Hawley, associate education commissioner, meaning that it raised familiar issues the department has heard before. “I also think it will be very interesting to see how people’s answers change” after having administered the spring tests. Hawley spoke with Chalkbeat Colorado on Tuesday.

The first-phase results were presented to the State Board during its Wednesday meeting in Grand Junction. The presentation marks the start of what’s expected to be several months of intense discussion about statewide testing.

The testing system
  • New online science tests were given in grades 5 and 8 this spring
  • The first social studies tests, also online, were given in grades 4 and 7
  • Science and social studies tests will be given in 12th grade next fall
  • New online CMAS language arts and math tests roll out next spring. Tests start in 3rd grade, and 11th grade tests are being added
  • There will be two sets of CMAS tests, one in early spring and one near the end of the school year
  • Schools administer annual literacy assessments in grades K-3 under the READ Act
  • Districts are phasing in school readiness assessments required by the CAP4K law

Not only is the second WestEd report coming, but a 15-member appointed Standards and Assessments Task Force is supposed to start work by July 15. Its assignment is to study the impact of testing on teaching time, the interaction of testing with the state accountability and educator evaluation systems and the feasibility of waiving some assessment requirements, among several other issues. (Get more information on the task force in this legislative staff memo.)

The legislation that created the task force also allocated $142,750 to CDE to coordinate the group’s work, commission a testing cost study, pay for a separate review analyzing how different testing schemes would affect the accountability system and obtain legal advice on the implications of letting parents and districts opt out of some testing requirements.

The task force and CDE are to report findings and recommendations to the legislature by next Jan. 31, giving the 2015 session plenty of time of consider the issue. District and parent anxiety about testing was an undercurrent during the 2014 session. But lawmakers rejected bills that would have delayed rollout of new tests next year and cut back on the new social studies tests. The task force bill was the only testing legislation passed. (That measure originally would have allowed districts to opt out of some testing requirements.)

CDE commissioned the WestEd study before the task force bill was passed.

Here are some other key conclusions from the WestEd survey and focus groups:

  • People are worried about the impact of testing on instructional time
  • Districts have only moderate levels of readiness for the new tests
  • Quantity, frequency and duration of assessments are a concern
  • Administrators, teachers and parents want timely, relevant and useful results
  • There’s concern about the burden and usefulness of high school tests
  • Respondents believe there needs to be greater recognition of local assessment systems and practices

Hawley said the survey found respondents want tests that take less time and return results more promptly, but that they also value the academic growth data produced by statewide tests. “That says to me they want shorter, timelier assessments that still measure student growth.”

The surveys also found that respondents generally value local interim tests more than state tests, dislike the high stakes attached to state tests and delayed results and perceive a lack of value for instruction in statewide exams.

School readiness tests and the social studies and science exams were seen as the most burdensome by large majorities of respondents.

The report notes that the results are weighted toward the concerns of smaller districts. Some 87 of 178 districts responded to the survey — 73 percent rural, 16 percent suburban and 8 percent urban. Just over half of respondents were from districts with fewer than 1,000 students.

Focus groups were conducted in the Archuleta, Buena Vista, Cherry Creek, Delta, La Veta, Platte Valley, Strasburg and Woodland Park districts, along with separate focus groups for teachers and parents.

But Hawley said because rural districts make up 80 percent of all Colorado districts, “We feel it’s representative.” She noted that the second part of WestEd’s work will seek to gather more opinions from metro-area and large districts. Those results will be available in July, just as the task force is starting up.

“Districts generally view current English language arts and mathematics assessments (i.e., TCAP) as low value with suburban districts valuing these assessments more than urban and rural districts. Two-thirds of rural and suburban districts view all TCAP assessments as high in burden compared to 29 percent of urban districts. … And although the burden of school readiness assessments is high, urban districts (100 percent) consider it highly valuable in informing student progress compared to 38 percent of rural and 13 percent of suburban districts,” according to the report’s executive summary.

The report also contained some interesting responses about districts’ readiness for new online tests.

“Regarding overall readiness to administer state assessments, 27 percent of districts appear fully prepared, 53 percent reported moderate readiness levels, and 20 percent are not yet ready. The two primary factors influencing district readiness are management (62 percent) and devices (60 percent). … Suburban districts appear to be the least prepared with 79 percent citing management and IT staff and personnel issues,” the report said.

Jill Hawley, Colorado Department of Education
Jill Hawley, Colorado Department of Education

Hawley agreed the main concerns about readiness involve staff time and management of testing and showed fewer worries about technical issues like availability of hardware and connectivity.

(The switch to online testing means that districts will have to spread assessments over a longer period of time so that classes can be rotated through computer labs to take the exams.)

The study found strong sentiment for slowing the move to new tests.

“The solution noted most frequently by focus group participants was that of holding schools and districts harmless until all components of the system are validated and functioning effectively. … Finally, because participants feel overwhelmed and under-resourced, they desire a more gradual pace and seek to slow the roll-out of the new assessment system.”

Arredondo reiterated that point Wednesday, saying, “The number one idea mentioned from focus groups was this idea of holding harmless.”

The study suggested four possible paths for state policymakers to consider as they plan for new tests:

  • Implement the transition plan as scheduled
  • Stay the course with added supports and policy adjustments
  • Purposefully delay parts of the system
  • Strategically eliminate specific assessments

The WestEd report also suggested some short-term options for consideration by the state, including:

  • A phase-in of online assessments and offering paper options
  • Providing emergency funds for districs to purchase devices
  • Reducing the number and length of test sessions
  • Use a sampling approach for social studies exams
  • Make the school readiness assessment optional
  • Make 9th and 10th grade language arts and math tests optional
  • Revert to federal minimum testing requirements and made all other tests optional
  • Hawley noted that while the surveys found wide agreement on testing issues, “There are not shared beliefs on the solutions.”

    All those suggestions are likely to be key points of discussion as the task force begins its work.

Testing Time

New York’s state test scores are coming out today. Here’s what we’ll be watching for.

PHOTO: Dylan Peers McCoy

At long last, New York state English and math test scores are being released today, according to state officials.

Though the tests have been criticized as providing only a snapshot of what students have learned, they are still one of the main tools used to judge the progress of schools, students, and major education initiatives.

Because the tests themselves held steady between 2016 and 2017, the test scores will provide a brief glimpse into whether students are making progress. That could also mean smaller changes than last year, when English scores shot up nearly eight percentage points.

But stable comparisons will only make a temporary appearance in New York education. Next year, the state has announced it will shorten testing by two days, which will no doubt call yearly comparisons into question again.

Here are a few storylines we’ll be watching as the state prepares to release test results:

Is the city’s approach to education policy working? What about signature programs like “Renewal”?

When test results jumped almost eight points in English and roughly one point in math last year, Mayor Bill de Blasio was quick to say they showed “pure hard evidence” his policies were working.

The state’s top education policymakers, however, cautioned that due to changes in the test, an “apples-to-apples” comparison to the year before was impossible. The changes included offering students unlimited time and shortening the exams slightly. This year, state tests were kept consistent in order to make those comparisons possible.

That raises an important question: Without changes to tests, will the results still be good news for Mayor Bill de Blasio and Chancellor Carmen Fariña? It is particularly important for high-profile efforts like the city’s “Renewal” turnaround program, which is entering the year in which the mayor said it should show results.

The only other year under de Blasio’s tenure when the state had fairly consistent testing compared to the prior year was 2015. In that year, test scores inched up one point in math and two points in English.

It may be a good thing if there are only small increases again, said Jennifer Jennings, an associate professor at NYU who has studied testing.

“We hope that they’re gradually inching upwards,” Jennings said. “Very large swings are often evidence that something is off.”

What will happen to opt-out?

For the last two years, about one in five students across New York state have boycotted state tests in protest. That number is significantly lower in New York City — though it has been growing. In 2016, 2.4 percent of city students sat out of the English exams and 2.8 opted out of math in New York City.

The opt-out rate acts as a litmus test of the public’s reaction to state education policy. The movement started in response to a series of state reforms, including adoption of the Common Core and test-based teacher evaluations.

Despite changes the state made last year to appease families upset about the tests, opt-out rates remained relatively consistent. (In fact, they ticked up a bit.) This year, the state has embarked on a process to reshape learning standards and submit a new plan to evaluate schools under the new federal education law.

Will that be enough to defuse some of the tension? Early results indicated opt-out rates may have decreased statewide, but the final tally will likely be released today with state test scores, as it has in past years.

What about equity, which is at the heart of the city’s agenda?

Each year, test results show a disheartening fact: Certain subgroups, such as black and Hispanic students, fare worse than their peers on tests. English learners and students with disabilities also historically score below average.

City and state officials have placed equity at the heart of their agenda. Mayor Bill de Blasio’s “Equity and Excellence” initiatives and his approach to turning around struggling schools are predicated on the idea that schools — particularly those in low-income communities — need resources in order to be successful. State officials have put equity at the center of their plan to reshape education policy under the Every Student Succeeds Act.

De Blasio’s critics, however, have argued the best way to address equity issues is tackling segregation in New York City schools. (The mayor released a preliminary diversity plan, but has been fairly slow to endorse integration as a strategy for school improvement.)

So, are the extra resources helping to close the gap between students of color and their white and Asian peers? This year’s test scores will help sort out that question, though many of the mayor’s major initiatives could take years to come to fruition.

On Close watch

State’s lowest performing schools and districts taking hard look at this year’s test data

PHOTO: Denver Post file
Sixth-grade science teacher Monica Wisniewski works with Pija Williams Terralee, left, and Myth Cubbison at Kearney Middle School in Commerce City. Kearney is in Adams County School District 14.

Testing data for Colorado’s longest-struggling schools and districts show mixed trends.

Results released Thursday are from exams students took last spring, before the State Board of Education approved corrective action plans for the five districts and a dozen schools that had run out of time on Colorado’s accountability clock.

The Pueblo City 60 district saw a decline across many tests and grade levels while Westminster Public Schools showed improvements in 10 of 14 English and math tests.

Those districts, like other districts and schools facing state intervention this past school year, were already making changes before their state improvement plans were finalized. Much of that work is incorporated into the plans.

Thursday’s test data will be used toward a new state rating, one which these districts and schools must improve soon. The state plans gave most schools and districts until 2019 to earn a higher quality rating for face potential consequencs. But some, including the Adams County School District 14 and Adams City High School in Commerce City, must have a higher rating by 2018.

Find your school’s scores
Search for your school’s growth scores in Chalkbeat’s database here, or search for your school’s test results and participation rates in Chalkbeat’s database here.

The district of almost 7,500 students saw some improvements, but still is posting very low scores. For instance, 9.3 percent of fifth-grade students met the state’s learning goals in math, up from 7.9 percent last year. In that area, the district did better than the state, as fewer fifth graders did well on math tests statewide than last year.

At Adams City High School, growth scores, which represent how much students learned in a year compared to similar-performing students, decreased for both math and English. The school had an interim principal for much of the school year, which led to a student walkout in the spring.

Overall, Adams 14’s proficiency numbers are still lower than state averages.

Of these low-performing districts, the Pueblo City 60 district, which faced state action only for some of its schools but not as a district, was the only one that had decreases in growth scores for both English and math tests.

In English, the growth score was 43, down from 47 last year. That means students this year scored on average better than just 43 percent of Colorado students who had similar test scores last year.

The Pueblo district saw an increase in how many students met or exceeded expectations in eighth-grade English. One possible reason officials pointed to: innovation schools granted flexibility from some rules and state laws.

Dalton Sprouse, a district spokesman, said district officials are relatively pleased with the improvements they see in the data, especially when broken down by school.

“Given that there’s just two years of growth data, some of the fluctuation could be expected,” Sprouse said. “We see this as we’re maintaining the progress we made last year.”

Sprouse noted that two of the three schools that faced the state board earlier this year for low performance saw big increases in the number of students meeting math expectations.

“Some progress is starting to take place,” Sprouse said. “The assessment office is already working with principals to really dig into that data.”

Westminster Public Schools, another district that faced state action this year and is now on a three-year improvement plan, saw more improvements than the other districts on the state watchlist.

“We are pleased to see our focus on high expectations and personalized learning for all students is paying off,” Superintendent Pamela Swanson said in a statement.

The Westminster district, however, was also one of the only districts in the metro area where English language learners had worse growth scores than native English speakers in both math and English. Last year, there was no gap in growth on English tests.

Last year, about 40 percent of students in Westminster schools were English language learners.

In Adams 14 schools, where about 46 percent of students are English language learners, those students posted higher growth scores than native English speaking students.

Westminster did increase their overall rate of growth according to median growth scores, and reached above 50 for English language arts.

Aurora Public Schools, the only district at risk of state action next year, posted increases and also got one growth score above 50, which is critical to catch students up when they are behind grade level.

Here’s how districts that ran out of time on Colorado’s accountability clock — or districts that had schools that did — compared: