Are Children Learning

Indiana seeks $4 million from testing company blamed for ISTEP screw-ups

PHOTO: Shannan Muskopf via Flickr

This story has been updated to reflect the most recent information available.

The Indiana Department of Education is seeking $4 million in damages from the company that created last year’s problem-plagued ISTEP test.

The state blames the California-based CTB company for the scoring problems and technical glitches that led to delays in releasing last year’s test results.

State Superintendent Glenda Ritz today told the Indiana State Board of Education that the state had sent a letter to CTB making the request for damages but has not yet received a response. Spokesman Daniel Altman said the state is working with the attorney general’s office, but a lawsuit has not yet been filed.

The letter, written by state lawyer Bernice Corley, notes that the state’s contract with CTB included penalties for each day the scores were delayed that would have added up to $11.5 million were it not for language in the contract that limited damages to about $2.3 million.

Corley argued that the state’s significant expenses warranted a $4 million payment.

“The $4 million number was arrived at with an understanding of what the cap of the contract is but also with an understanding of the amount of damage that the department itself went through,” Altman said.

Corley wrote that CTB was expected to deliver score results by September 2015, but didn’t get them to the state until October.

“While the contract caps liquidated damages … that amount cannot begin to make Indiana whole,” Corley wrote. “Accordingly, IDOE demands $4M in damages from CTB for failure to timely deliver (test results), as well as the delay caused by the rescore in full and final resolution of all disputed issues between IDOE and CTB.”

Scores for the 2015 ISTEP scores were delayed in part because of reported problems with grading new computer-enhanced questions that allow students to manipulate the information on screen in ways that were impossible on prior tests.

Those scoring problems ultimately derailed the entire scoring process, delaying the release of exam results. The delay forced the state to postpone the release of A-F school accountability grades and to bar, for one year, the use of student test results in evaluating and paying teachers.

The Indiana General Assembly passed “hold harmless” legislation swiftly during the first few weeks of the legislative session in January.

“The delay was so disruptive to Indiana that the General Assembly had to take action during the legislative session following the administration of the ISTEP+ test to limit harm to teachers who were at risk of not receiving a performance award,” Corley wrote.

This was the fourth time for which ISTEP issues can be traced back to problems at CTB. In April of 2013, 16 percent of all Indiana students taking ISTEP, about 78,000 kids, experienced interruptions during their tests. That year, letter grades weren’t released until December.

In 2011 and 2012, about 10,000 and 9,000 students respectively had online testing issues. Because of the interruptions in 2013, the state and CTB came to a settlement for $3 million.

Altman said the state currently does not have any testing contracts with CTB.

Chalkbeat reached out to CTB’s former parent company, McGraw-Hill Education, about the letter, but Catherine Mathis, the group’s chief communications officer, said officials declined to comment.

Find more Chalkbeat stories on these topics below:

high-stress testing

How the stress of state testing might make it harder for some students to show what they know

PHOTO: Anthony Lanzilote

The annual ritual of state testing in elementary and middle schools often comes within an unwelcome side effect: jittery, stressed-out kids.

Now, a first-of-its-kind study documents some of what’s actually happening to students.

It found that students in one New Orleans charter network saw modest spikes in cortisol, a hormone caused by stress, leading up to state exams. And the students whose cortisol spiked most or crashed furthest did worse than predicted — suggesting that the test scores reflect not just what students know, but how they perform under pressure.

The five researchers behind the study call that a “stress bias.” The paper finds some evidence that students living in higher-crime, higher-poverty neighborhoods are most affected.

That’s not surprising from a biological perspective, said Pamela Cantor, a psychiatrist and the founder of Turnaround for Children, a group that works to address the effects of trauma on children in schools.

“What we’re in effect doing to kids who are exposed to adversity on a chronic basis is actually putting them in a highly unfair situation, where their biology may overreact to the stress and not give them a very good opportunity to reveal the things they likely know,” she said.

The research looks at fewer than 100 students, and some of the findings are ambiguous. “I don’t want to make broad policy claims based on this one paper with a relatively small sample size in one setting,” said lead author Jennifer Heissel, a professor at the Naval Postgraduate School. But, she said, “if this is replicated in other settings for other students, we need to reconsider perhaps what we are using high-stakes tests for.”

The study, released earlier this month through the National Bureau of Economic Research, focuses on a network of three charter schools in New Orleans. The researchers analyzed an unusual data source: saliva samples of 93 elementary and middle students, obtained with parents’ consent, during the 2015-16 school year.

The research team compared cortisol levels of students at three points: during a regular week, a week when students took a low-stakes practice test, and the week students took the state test.

Cortisol is a hormone that generally increases after someone wakes up and declines from there. It jumps in response to stress or challenges and decreases due to boredom or disengagement.

The researchers focus on cortisol levels in the period right before the exam, when students were likely to be most be stressed about testing. Indeed, cortisol levels were about 15 percent higher at that time during testing week than they were during a regular week.

“That is in line with other stressors you might encounter through your day,” Heissel explained.

But some students responded more dramatically. “On average, there’s this increase, but individual kids are going in different directions,” she said.

Students whose cortisol noticeably spiked or dipped tended to perform worse than expected on the state test, controlling for past grades and test scores. Boys saw bigger changes than girls. and so did students from higher-poverty neighborhoods, though this difference was not statistically significant. (Keep in mind that the students in the study were almost all low-income, so there was limited room for comparison.)

Source: “Testing, Stress, and Performance: How Students Respond Physiologically to High-Stakes Testing”

It’s possible that some students’ jumps or dips in cortisol during testing week were due to other factors in their lives. But Cantor of Turnaround for Children said it’s not surprising that tests would induce stress, or that students more likely to have experienced trauma would respond differently.

“For children who face adversity in a chronic way, that system is pumped and primed much more so than other kids,” Cantor said. “If a child does overreact to a trigger, one of the ways that manifests itself is that they shut down, they freeze.”

Is that a “stress bias”? If tests unfairly penalize students who respond poorly to stress, that might suggest that “tests aren’t fully capturing what we want and perhaps what we are thinking that they capture,” said Heissel.

Another interpretation, she noted, is that the ability to perform well under pressure is part of what exams measure, and that it’s a skill “to be able to wrangle your stress response.”

The results raise a number of unanswered questions.

One is whether the results would hold for in-class exams administered by teachers. In many cases, those tests have higher stakes for students than do state exams, which in New Orleans have been used to grade and in some cases close schools.

Another is whether the way students are prepared for state exams might affect their stress. The paper offers limited information on the charter network being studied, which is anonymous. But a number of high-profile charter schools place substantial emphasis on preparation for state tests. It’s unclear whether this approach increases or reduces test-related stress.

Either way, the latest paper suggests one way to produce better scores is to ensure students stay calm during testing.

And a final question is, how else might the stress of testing affect students? Past national research offers mixed evidence on whether No Child Left Behind, the federal education law that led to the current state testing regimen, led to general increases in student anxiety.

Cantor said that the key to buffering against adversity is having warm, positive relationships, which can prompt the release of anti-stress hormones.

“A teacher who communicates belief and confidence and inspires trust in kids — that teacher is activating a hormonal system that opposes the effects of cortisol,” she said.

Measure of Success

State ratings identify 163 Colorado schools in need of improvement

PHOTO: AAron Ontiveroz/The Denver Post
Students in kindergarten on the first day of school at McGlone Academy.

More than 160 Colorado schools received one of the state’s two lowest ratings, making them eligible for additional assistance but also vulnerable to intervention if they don’t improve student performance.

The watch list comprises 9 percent of Colorado’s 1,800 schools and educate roughly 74,000 students, or 8.5 percent or the state’s almost 900,000 students. That means the vast majority of students in the state attend a school with one of the two higher rankings on the four-point scale.

The State Board of Education finalized the ratings Wednesday. The state gives separate district-wide ratings, which were finalized last month.

“The state’s accountability system is built on the premise that all students should receive a high quality education and graduate ready for college or careers,” Katy Anthes, Colorado’s education commissioner, said in a statement. “Our goal is to give all students a chance to excel. These designations allow us to identify struggling schools that may need more support to help students achieve their highest aspirations. And they also highlight successful schools so that other schools can learn from them.”

All public schools receive a state rating, known as the School Performance Framework report, each year. It’s based largely on student scores on the state’s English and math tests. Student growth, or how much students learn year-to-year compared to peers with similar results on state tests, carries most weight. High school graduation and dropout rates are also factored in.

There are four ratings: performance (the highest), improvement, priority improvement and turnaround (the lowest).

Schools and districts that have one of the lower two ratings are placed on a watch list and have five years to improve before facing state intervention. Schools on the list are eligible for grants for leadership training and help from outside consultants, but if change doesn’t come fast enough, the state could hand over control to an external manager, require conversion to a charter, or close schools.

Earlier this fall, the State Board of Education ordered the Adams 14 school district, based in Commerce City, and two schools in Pueblo in southern Colorado to turn over control to external managers after earlier intervention efforts did not produce enough improvement.

Colorado is still figuring out what effective intervention looks like and if outsiders can make a difference for students that existing leadership has not been able to achieve.

Most Colorado schools maintained the same rating they had in 2017, with 15 percent moving down at least one level and 14 percent moving up at least one level. Eighteen schools improved enough to get off the state watch list, which is often known as the “accountability clock,” some after initial state intervention last year.

Six schools are entering their eighth year on the watch list: Aurora Central High School, Adams City High School, Aguilar Junior-Senior High School in the tiny Aguilar district in southern Colorado, Hope Online Learning Academy Elementary School in Douglas County, Heroes Middle School, and Risley International Academy of Innovation, the last two both in Pueblo.

Two are entering year six: Central Elementary School in the Adams 14 district and Minnequa Elementary School in Pueblo.

Another four are entering year five, now the last year to improve before state intervention: Manual High School and Montbello Career and Technical High School in Denver, Mesa Elementary in the Montezuma-Cortez district in southwest Colorado, and EDCSD: Colorado Cyber School in Douglas County.

In the past, some schools received more time to improve because the “clock” was paused for several years as the state changed assessments. But now there are no more extensions beyond year five.

Colorado Department of Education

Of the state’s 42 online schools, a little more than half received one of the top two ratings, and 31 percent did not report enough data for the state to grant a rating. Colorado has more stringent regulations of online schools than many states, but there is an ongoing debate about how well these schools serve students.

About 84 percent of the state’s 247 charter schools received one of the top two ratings, compared to 89 percent of all Colorado schools. Twenty-six charter schools, or 10.5 percent, received one of the lowest two ratings.

Look up your school here: