Are Children Learning

CTB: More ISTEP problems will delay results for months

PHOTO: Alan Petersime

Stop us if you’ve heard this one before: ISTEP scoring problems will cause what could end up being a months-long delay for the release of scores and assigning of school A-to-F grades.

For the fourth time since 2011, Indiana faces problems with the administration of its state exam, overseen by California-based CTB, formerly CTB/McGraw-Hill. Past problems were a major factor in the state’s decision to switch to British-based Pearson starting next year.

Indiana will cut ties with CTB at the end of its four-year, $95 million contract once it delivers this year’s results. But company president Ellen Haley told the Indiana State Board of Education today that won’t come as soon as expected.

Haley, who has made regular appearances over the years to publicly apologize to Indiana officials for test problems, this time blamed the complications on new computer-enhanced questions the state asked to be included on ISTEP that allow students to manipulate the information on screen in ways that were impossible on prior tests.

Steve Yager, a board member and former superintendent of Northwest Allen County Schools in Fort Wayne, was scathing in his response to Haley’s explanation of the scoring problems. He said he has little faith the company can actually make good on it’s promises.

“You stand here and say you’ll deliver this, but I’ll believe it when I see it,” Yager said. “What’s happening is girls and boys are just being damaged, and teachers are being damaged, by the ineffective practices of your company.”

Company blames Indiana’s new standards

State Superintendent Glenda Ritz said she also is frustrated by the delays, which could mean letter grades aren’t finalized until early next year, but said there was little the state could do other than wait for the company to re-score the test questions to ensure the results are correct.

“I also want to make sure that kids are getting credit for everything on their test,” Ritz said. “And since it is the first time for technology-enhanced items, we just have to make sure we have it absolutely right.”

Haley blamed the problems on Indiana’s decision to institute new academic standards last year and then quickly adapt its exams to match up with them. As a result, she said, the state could not try out test questions on students before the test was given this past spring, which normally would have been the process.

“This is the nature of switching so dramatically,” Haley said. “It’s a good thing — new standards, and a new test — but there’s nothing leftover from the previous test. You’re starting all over again.”

Cari Whicker, a board member and teacher from Huntington, had little sympathy for the testing company.

“I’m sorry it’s more work for you,” Whicker said. “But you know what? It’s been a lot more work for people in the field.”

Other board members argued the problem could have been avoided. Instead, another year of problems only erodes further the shaking confidence that test results can be trusted, they said.

“Many of our teachers and principals and parents and students aren’t having confidence because of all the complications from last spring,” board member Lee Ann Kwiatkowski said. “And now with having another timeline being pushed back further, we’re going to once again reduce the confidence they currently have when they get results.”

Re-scoring to hold up ISTEP scores, school letter grades

Technology and student creativity are what made grading the new test questions so difficult, Haley said, arguing the company isn’t to blame for the delays.

New test questions were given for the first time when students took the test in March, Haley said, and the guidelines the company created for scoring didn’t recognize all of the possible correct answers students gave for the new technology-enhanced questions. In the past there was just one path to a correct answer. Now students can get questions right in different ways, such as by manipulating graphs or writing equations.

“They came up with the correct answer, but they responded differently,” Haley said. “If we don’t pause and change rubric, the student doesn’t get credit for that answer.”

Board member Vince Bertram, a former Evansville superintendent and executive director of Project Lead The Way, said a company that does so much testing work across the country should have anticipated all possible answers. Bertram’s company offers schools project-based curriculum in science, technology, math and engineering fields.

“How could we miss this?” Bertram said. “Don’t we know how a fifth-grader is going to answer a math question?”

Testing director Michele Walker estimated that when time to re-score is factored in, school letter grades might not be released until January or February, at the latest. Schools would still get students’ ISTEP grades by the end of this year.

Company has history of problems in Indiana

This is the fourth year out of five for which ISTEP issues can be traced back to problems at CTB.

In April of 2013, 16 percent of all Indiana students taking ISTEP, about 78,000 kids, experienced interruptions during their tests. That year, letter grades weren’t released until December.

In 2011 and 2012, about 10,000 and 9,000 students respectively had online testing issues. Because of the interruptions in 2013, the state and CTB/McGraw-Hill came to a settlement for $3 million.

Haley said repeatedly during the meeting that delays are part of the process when tests and standards change, especially if those tests involve new technology. Other states are dealing with the same issues, she said.

But other CTB customers, such as Oklahoma, have also suspended their work with work with the company.

Are Children Learning

Second study shows Indianapolis charter students fare better on tests

PHOTO: Dylan Peers McCoy

The second study in a week shows strong test scores for students at Indianapolis charter schools, bolstering the claims of advocates in a city where school choice continues to expand.

Indianapolis elementary students who attend mayor-sponsored charter schools beginning in kindergarten — and remain in the same schools — make bigger improvements on state tests than their peers in traditional schools across the city, according to the latest study.

The study contributes to emerging research that suggest that charter schools that are well managed and have good instruction can be successful, said co-author Hardy Murphy, a clinical professor of educational leadership and policy studies at the IUPUI School of Education.

The results of the study indicate Indianapolis charter school students are doing better than they would’ve done if they hadn’t enrolled in charter schools, Murphy said.

“This does not appear to have happened by chance,” he said. “I believe that the school experiences and the instructional teachers of those schools they are enrolled in are actually a big part of the results that we are seeing,”

The educational landscape in Indianapolis is defined by school choice. About 18,000 students who live in Marion County attend charter schools, and thousands more transfer to nearby districts or attend private schools with vouchers, according to state data. In recent years, the state’s largest district, Indianapolis Public Schools, has also become a national model for partnerships with charter schools. That makes understanding school performance essential for parents — but unpacking whether schools actually help boost student achievement can be particularly thorny for researchers.

With this study, Murphy said he and co-author Sandi Cole, director of the Center on Education and Lifelong Learning at Indiana University Bloomington, hope to disentangle one factor that makes studying charter schools difficult: the dips in test scores that students often experience after transferring to new schools. Murphy’s research focuses on students who began in charter schools in kindergarten and compares them to similar students in traditional public schools in Indianapolis.

“It’s time to move beyond the debate about whether or not charter schools are effective and start talking about, when they are effective, why, and for whom?” Murphy said, adding that successful approaches can be used in other settings.

The study focuses solely on students who attend charter schools authorized by the mayor’s office. For the control group, the study included township districts as well as Indianapolis Public Schools. The researchers plan to present their results to the education committee of the Indianapolis City-County Council and the 2019 Conference on Academic Research in Education.

The findings add to a growing body of research on Indianapolis charter schools. Last week, the Stanford-based group CREDO released a report that found that students at charter schools had test score gains that mirrored the state average, while Indianapolis Public Schools students made smaller gains on math and reading tests than their peers across the state. Another recent study found that when students moved to charter schools their test scores held steady.

To Do

Tennessee’s new ed chief says troubleshooting testing is first priority

PHOTO: (Caiaimage/Robert Daly)

Penny Schwinn knows that ensuring a smooth testing experience for Tennessee students this spring will be her first order of business as the state’s new education chief.

Even before Gov.-elect Bill Lee announced her hiring on Thursday, she was poring over a recent report by the state’s chief investigator about what went wrong with TNReady testing last spring and figuring out her strategy for a different outcome.

“My first days will be spent talking with educators and superintendents in the field to really understand the scenario here in Tennessee,” said Schwinn, who’s been chief deputy commissioner of academics in Texas since 2016.

“I’ll approach this problem with a healthy mixture of listening and learning,” she added.

Schwinn’s experience with state assessment programs in Texas and in Delaware — where she was assistant secretary of education — is one of the strengths cited by Lee in selecting her for one of his most critical cabinet posts.

The Republican governor-elect has said that getting TNReady right is a must after three straight years of missteps in administration and scoring in Tennessee’s transition to online testing. Last year, technical disruptions interrupted so many testing days that state lawmakers passed emergency legislation ordering that poor scores couldn’t be used to penalize students, teachers, schools, or districts.

Schwinn, 36, recalls dealing with testing headaches during her first days on the job in Texas.

“We had testing disruptions. We had test booklets mailed to the wrong schools. We had answer documents in testing booklets. We had online administration failures,” she told Chalkbeat. “From that, we brought together teachers, superintendents, and experts to figure out solutions, and we had a near-perfect administration of our assessment the next year.”

What she learned in the process: the importance of tight vendor management, including setting clear expectations of what’s expected.

She plans to use the same approach in Tennessee, working closely with people in her new department and Questar Assessment, the state’s current vendor.

“Our job is to think about how to get online testing as close to perfect as possible for our students and educators, and that is going to be a major focus,” she said.

The test itself has gotten good reviews in Tennessee; it’s the online miscues that have many teachers and parents questioning the switch from paper-and-pencil exams. Schwinn sees no choice but to forge ahead online and is quick to list the benefits.

“If you think about how children learn and access information today, many are getting that information from hand-held devices and computers,” she said, “so reflecting that natural experience in our classrooms is incredibly important.”

Schwinn said computerized testing also holds promise for accommodating students with disabilities and provides for a more engaging experience for all students.

“When you look at the multiple-choice tests that we took in school and compare that to an online platform where students can watch videos, perform science experiments, do drag-and-drop and other features, students are just more engaged in the content,” she said.

“It’s a more authentic experience,” she added, “and therefore a better measure of learning.”

Schwinn plans to examine Tennessee’s overall state testing program to look for ways to reduce the number of minutes dedicated to assessment and also to elevate transparency.

She also will oversee the transition when one or more companies take over the state’s testing program beginning next school year. Former Commissioner Candice McQueen ordered a new request for proposals from vendors to provide paper testing for younger students and online testing for older ones. State officials have said they hope to award the contract by spring.

In Texas, a 2018 state audit criticized Schwinn’s handling of two major education contracts, including a no-bid special education contract that lost the state more than $2 million.

In Tennessee, an evaluation committee that includes programmatic, assessment, and technology experts will help to decide the new testing contract, and state lawmakers on the legislature’s Government Operations Committee plan to provide another layer of oversight.

Spring testing in Tennessee is scheduled to begin on April 15. You can learn more about TNReady on the state education department’s website.

Editor’s note: This story has been updated with new information about problems with the handling of two education contracts in Texas.