Ready or not, online tests coming to Colorado

Colorado kids will get a chance to put their computer skills – and their academic knowledge – to high-stakes use when the state’s first online tests roll out later this school year for some students.

The plan is for the tests to be piloted this year followed by statewide testing in 2015.

So by the spring of 2015 more Colorado students will be taking more tests than under the current TCAP system, and they will be taking those tests on computers and tablets.

And, those students likely won’t be scoring as well as they did on the TCAPs, given that the new tests, partly based on the Common Core Standards, will be testing different things and are supposed to be harder than exams given in the past.

In the meantime, administrators and tech staff in districts across the state are fretting as they try to gear up for a whole new testing environment. Districts are worried about whether they have enough bandwidth and enough computers, but some administrators are even more concerned about the planning and tight scheduling that will be needed to fit online tests into school calendars.

“The general state of readiness is – we’re probably at a C-minus,” said Steven Clagg, chief information officer for Aurora Public Schools and president of the Colorado Association of Leaders in Education Technology. Clagg tempers his remark by noting his impression is based on anecdotal evidence, adding, “I think the state of Colorado is going to make it through this, but it won’t be without pain.”

The first wave of about 240,000 students will face new tests in April 2014 when students in four grades will take social studies or science tests. High school seniors will take both of those tests in the fall of 2014.

And, in the spring of 2015, all Colorado students in grades three through 11 will take online tests in language arts and math. Those will replace the TCAP reading, writing and math tests currently given to students in third through 10th grade. The new tests are being developed by the Partnership for Assessment of Readiness for College and Careers (PARCC).

The social studies and science tests are launching during the same school year that districts are implementing new content standards on which the PARCC tests will be based, rolling out upgraded early literacy programs for students in kindergarten through third grade and evaluating all principals, assistant principals and teachers using systems based on the landmark 2010 educator effectiveness law. All of those mandates have ratcheted up the level of anxiety in school districts statewide. (See this EdNews story about the new evaluation systems.)

Here’s a look at the some of the features of and the issues raised by the new tests.

Testing the tests

The science and social studies exams were field tested last April in 261 schools, where 25,408 sample tests were given.

“We were very pleased with how that spring field test went,” said Joyce Zurkowksi, Colorado Department of Education executive director of assessment. “Our small districts did fine.” Clagg agrees that the spring tests provided valuable lessons.

The Hanover district southeast of Colorado Springs participated in the field test. Paul McCarty, superintendent of the 227-student, two-school district, said the experience highlighted the challenges of online testing.

McCarty’s district doesn’t have a technology staffer and has to rely on help from the area board of cooperative educational services. He’s worried about smaller districts like his having to rely on staff members who don’t have the appropriate tech background.

More sample tests will given this autumn, at elementary and middle schools during the second half of September and in high schools starting in late October.

Short sample tests also are available online (check them out here), and CDE is encouraging schools to have students practice on those tests so they can become familiar with the look and feel of the exams.

Who’s tested and when

The TCAP reading, writing and math tests, like the CSAPs before them, have been given every spring to students in the third through 10th grades. Students have taken science tests in grades five, eight and 10. Last spring about 500,000 students took a total of about 1.7 million exams.

That schedule is shifting.

The social studies tests will be given in the fourth and seventh grades. While science will continue to be tested in fifth and eighth grades, 10th graders will be off the hook. Instead, social studies and science tests will be given to high school students in the fall of senior year, the first time that statewide tests have been given to 12th graders.

Next spring will mark the first statewide testing on social studies; science tests have been around awhile but previously have been given on paper.

The new PARCC tests will be expanded to include 11th graders, who also will continue to take the ACT college-readiness test.

What the tests will be like

The biggest change for students, of course, will be taking tests on computers rather than with pencil and paper.

Tests will include multiple choice items, some of which will require students to move suggested answers around on the screen, as well as questions that involve reading a text and writing an answer.

The timing of PARCC tests also will be different from the TCAPs. The new exams will be given in two segments, one about three-quarters of the way through the school year and the second closer to the end of the year. Districts will be allowed “testing windows” of a few weeks intended to give them enough time to test students with the equipment available.

Students also are likely to spend a bit more time on the PARCC tests than they do on TCAPs. (See this CDE chart for a comparison.)

It’s estimated that total PARCC testing time will be eight hours for third graders, just over nine hours in grades four and five, a little less than nine and a half hours in middle school and just over nine and a half hours in high school.

Clagg and others believe the new tests will require a whole new approach by teachers and administrators.

For instance, he said, “Proctoring is going to be a lot different. … We believe you’re going to have to have two people in a room or maybe more.”

He also noted that computers often are packed closely in school computer labs. “You’re going to have kids shoulder to shoulder taking this tests.” He said that during testing this spring students sometimes looked at neighboring screens. “They weren’t trying to cheat. They just got distracted for a minute” by movement or a flash of color on a neighboring monitor.

So, Clagg said, proctors will have to learn a new set of skills, including restarting computers when they lock up, to effectively monitor online testing.

He also said the school tech staffers – or administrators in districts without the luxury of IT experts – will have to learn how to prepare computers ahead of time. Automated processes like an anti-virus scan will have to be turned off, he said, because they will lock a test if they fire up while a student is working.

Clegg also said schools will have to get highly organized about test scheduling in order to make the most efficient use of the computers available.

Hanover’s McCarty is worried more about scheduling than he is about tech issues like bandwidth.

“I think the scheduling of classes is huge,” he said. “We don’t have that many computers, and we try to use and teach and use technology as much as we can.” He noted that reserving a computer lab for testing will make it unavailable for instructional use.

Clagg echoes that concern. “What about the classes that were going to use that lab? A lot of the challenge is going to be, what’s the ripple effect of the test? … We’re going to have to be really creative in the districts.”

Will school districts be ready?

Zurkowski believes school districts will be ready for next spring’s social studies and science tests, which she said “will help us gauge what’s needed for PARCC. …We are really expecting that by 2015 we will have worked through some of the hiccups.”

Based on data compiled from a sample of districts, CDE estimates 43.1 percent of middle and elementary schools can complete the science and social studies tests in less than five days, and 51.4 percent can do that in six to 10 days.

The department also estimates 85.5 percent of high schools can do 12th grade testing in under five days and that 10.5 percent can do so in six to 10.

Next spring’s tests are being provided by Pearson Education, one of the nation’s largest testing companies. Zurkowski said Pearson also is expected to provide the PARCC tests, so similarity in test format and technical specifications should be an advantage for Colorado.

There’s been a lot of concern about districts’ access to high-speed Internet connections – bandwidth – but Zurkowski believes one Pearson technology should ease some of those concerns.

The company offers what’s called “proctor caching,” which means districts can download full sets of test materials and that students can take the exams on a local network and not have to rely on a fast Internet connection. Once testing is finished the completed tests are transmitted back to Pearson.

“The proctor caching is going to help us a lot in this state,” she said. For instance, CDE estimates that in a sample district only 76 tests could be taken over the Internet simultaneously but that more than 1,000 machines could access a test on a local district network.

Clagg agrees that caching will help districts but notes, “It takes a bit of knowledge to get it implemented.”

Beyond bandwidth, Clagg notes that schools face other challenges. “Do districts have enough devices?”

CDE has calculated generous testing windows to account for limits on the number of computers that districts have. Clagg thinks those windows are good, but “the key thing is going to be scheduling.”

Asked if districts plan to buy new equipment for testing, Clagg said, “Everybody’s thinking about that.” Testing “is making us ask harder questions about the devices we’re buying. We have to look at multipurpose devices.”

McCarty said he thinks Hanover is ready for spring 2014 – if everything goes right – but that the PARCC tests in 2015 present a different challenge.

“At some point we’re going to have to set aside more money for computers and convert another classroom to a computer lab.”

The cost of new tests

PARCC announced last month that the estimated cost of its English and math tests would be $29.50 per student, just below the current median cost of $29.95 for states participating in the multi-state group.

Zurkowski said that’s “pretty comparable to what Colorado is used to,” citing a figure of $29.25.

The department’s 2014-15 budget request, which will be considered by the 2014 legislature, is expected to include $3.4 million in additional funds for the PARCC tests. Zurkowski said that covers addition of 11th grade tests and a bit extra to pay for paper tests for districts that may not be ready for online. (The PARCC paper tests come to about $34 per student.)

CDE also is likely to request an additional $300,000 for increased costs of ACT and English language proficiency tests, plus $3 million for early literacy assessments.

The state currently budgets about $34 million a year for all testing costs, including the TCAPs, science and social studies, work on “augmentations” to the PARCC tests, development of Spanish literacy assessments, alternate assessments for students with significant cognitive disabilities, ACT tests and ACCESS, an English language proficiency assessment. (The federal government provides about $7.6 million of the testing budget.)

Districts long have complained that they also bear the burden of testing costs, but it’s hard to put numbers on that.

Teachers and other professionals, of course, are on the payroll whether they’re proctoring tests or giving normal classroom lessons.

But there are other costs as well. A cost study done for the plaintiffs in the Lobato v. State school funding lawsuit estimated that schools, depending on size, bear non-personnel assessment costs of between $25 and $50 a student. Zurkowski said CDE hasn’t researched district testing costs.

Asked about district costs for the coming new tests, Clagg said, “I have not come up with a forecasted cost for this [but] there is going to be a cost for districts,” including for training.

The bigger issue, Clagg said, is that “The cost is an opportunity cost. What aren’t we doing so we can do this?”

New tests equal lower achievement

One of the final slides in a CDE presentation at a recent Colorado Association of School Executives convention gives this gives this understated warning:

“Given the increased rigor of both the Colorado Academic Standards and the new assessments, it is reasonable to prepare for a drop in performance based on them compared to the performance demonstrated under the current system.”

Department officials have been sounding that warning for months in a variety of meetings, and it’s commonly accepted that average performance drops when tests are changed, particularly when new exams test new skills.

Proficiency levels dropped in New York State after students took new tests based on the Common Core Standards. (See this Gotham Schools story for details.) Kentucky had a similar experience last year after students took new tests.

Tracking and using the data

While achievement levels are expected to drop, results on old and new tests aren’t necessarily comparable because they’re based on different standards, CDE officials say.

Past TCAP results “are going to become irrelevant in a very short period of time,” Zurkowski said. “Can we compare those numbers?” – referring to percentages of students scoring proficient or advanced. “Probably not.”

CDE officials are more concerned about the Colorado Growth Model, which calculates student test score growth over multiple years to determine if students are making sufficient progress. Growth is a key factor in the state’s accountability and rating system of districts and schools.

Alyssa Pearson, director of CDE’s accountability and data analysis unit, said the department is working on how to make the growth model bridge TCAP results and the new tests. “You can do it,” she said. Pearson noted that Colorado recently switched English language proficiency tests and was able to track most growth data across the two sets of data.

But, she said, calculating one key piece of growth data – whether a student is making adequate growth toward proficiency – probably will require two years of data from the new tests.

What if new tests aren’t ready?

The PARCC tests – and the Common Core Standards on which they’re based – have come under fire recently in some states. (See this EdNews story.)

During a recent State Board of Education budget session, chair Paul Lundeen asked if CDE should set money aside to pay for other tests if the PARCC exams aren’t ready in 2015.

Education Commissioner Robert Hammond, who sits on PARCC’s board, said, “So far my belief is they will make it.”

If that doesn’t happen, Colorado has other options, Associate Commissioner Jill Hawley told the board. “We may some options; we could still procure tests with another vendor. … The potential for getting something off the shelf exists.”