Are Children Learning

Is Indiana's state ISTEP exam too easy?

PHOTO: Shannan Muskopf via Flickr
State officials are closing as many 38 Michigan schools with low rankings due to test scores but they might have trouble finding higher scoring schools nearby

Indiana’s standardized tests might have gotten so much harder last year that test scores plunged across the state, but two national testing experts say the exam might technically still be too easy according to an earlier study.

“The Indiana test was relatively low-level,” said Ed Roeber, a former Michigan testing director asked to consult on ISTEP.

The 2015 ISTEP exam might have seemed tougher to students and teachers. The state saw a 20 percentage point drop in students passing both English and math last year.

But when Roeber and another testing expert, Derek Briggs, were asked by the Indiana State Board of Education to conduct a review of that exam in January, they discovered that exam questions were not as rigorous as they should have been. The state adopted new academic standards in 2014 that were designed to better prepare kids for college and careers. The new standards were said to be much tougher.

Try it out: 18 practice questions to prepare for Indiana’s 2016 ISTEP test.

Indiana officials brought on Roeber and Briggs on to advise them on the ISTEP last year in the wake of scoring glitches, delays and accuracy questions that raised concerns about the validity and reliability of exam. After examining an earlier study of the test by the education research group WestEd, the pair noted that Indiana’s test questions do accurately reflect the standards, but suggested that they might still be too easy.

The analysis from Roeber and Briggs reached no conclusions on whether the standards themselves are meeting the state’s goals, but the pair found that too many questions on last year’s ISTEP test were too easy.

Specifically, the experts cited the WestEd study that found a majority of the questions on the 2015 exam were fact-based questions that asked students to merely to recall information rather than do higher-level thinking. More than 80 percent of questions on the English exam and every single question on the math test were found to be low-level basic questions.

The test had very few more difficult questions that would have required students to describe or explain a scenario, use evidence to back up answers or test a hypothesis and make connections beyond the facts at hand.

“The fact that (a test question) aligns with the standards is good, but it doesn’t imply that it’s measuring the standards to the same depth the standard is written at,” Roeber said.

What remains unclear is why the questions were too easy. The fault could rest with the test, which was written hastily after lawmakers voted to drop the Common Core and write Indiana-specific standards in 2014. Test questions weren’t tried out on students until the test was actually given last spring — a highly unusual practice in a testing industry in which test questions are typically piloted over several years before being used on an exam, Roeber said.

The Indiana Department of Education decided against using questions from previous tests that could have saved some time, said Cynthia Roach, the state board’s testing director.

Another possible reason for the easy test questions is the state’s standards themselves. It’s not clear just how rigorous Indiana’s standards really are. Until the standards are properly evaluated using the same metrics as the test, Roeber said, the state’s policymakers can’t really know if ISTEP is measuring what kids are expected to learn better than past tests.

It might seem technical, but the mismatch between the standards and the questions could be important. Indiana still must submit its test for review by the U.S. Department of Education for its waiver from the No Child Left Behind law, still in effect until August. Any misalignment between the test and standards could be problematic.

“That’s one of the key things that the peer reviewer will look for is the measure of the rigor of the test, and if this misstates that, then you definitely have the right to challenge the vendor,” Roeber said.

The state’s new testing advisory committee is taking the suggestions from the analysis to try to move forward, Roach said. Nothing can be addressed in time for the 2016 test, which students have already started taking, but questions should be reassessed and Indiana should look into deeper measurement of its standards, Roeber said.

The Indiana Department of Education, which administers the test, declined to make its testing director available for an interview to discuss Roeber and Briggs’ analysis, but Department spokeswoman Samantha Hart issued a statement saying the test was hard enough.

“As with any assessment, there are going to be questions that are more and less rigorous than others — The new ISTEP+ exam is no different,” Hart wrote. “Anyone who thinks that last year’s ISTEP wasn’t hard enough should go talk to a student or a teacher or a parent. This test was clearly more rigorous, just like our standards.”

making the rounds

Tennessee’s new education chief ‘very confident’ that online testing will be smooth in April

PHOTO: Shelby County Schools
Tennessee's new education commissioner Penny Schwinn (second from left) met with Douglass High School students and Shelby County Schools leaders Friday.

As Tennessee’s new education commissioner wrapped up her second week on the job by visiting four schools in Shelby County, Penny Schwinn said she feels “very confident” the state has learned from its mistakes in online testing.

During the more than three-hour ride to Memphis on Friday, Schwinn said she continued to pore over documents showing evidence that the corrections the state department staff have put in place will work.

“I feel very confident that our team has looked into that,” she told reporters in a press conference after meeting with students. “They’re working with the vendor to ensure that testing is as smooth as possible this year.” Currently the state is working with Questar, who administered TNReady online last year.

She also said the state’s request for proposals from testing vendors, which is already months behind, will be released in about two weeks.

PHOTO: Shelby County Schools
From left: John Bush, principal of Douglass High School; Penny Schwinn, Tennessee Education Commissioner; and Joris Ray, interim superintendent for Shelby County Schools.

“No later than that,” she said. “We hope and expect to have a vendor in place before the end of the fiscal year,” in late June.

The day Schwinn was hired, she said getting state testing right would be her first priority. Three years of major technical failures have severely damaged the trust educators and parents have in the state’s test, TNReady. It is the main measure of how schools and teachers are doing, but state lawmakers exempted districts from most testing consequences in 2018.


From Schwinn’s first day on the job: Tennessee’s new education chief wants to ‘listen and learn’ with school visits


Prior to talking with reporters, Schwinn said she heard “hard-hitting questions” from several students at Douglass High School in Memphis about what the state can do to improve education. Schwinn has said she will visit Tennessee schools throughout her tenure to ‘listen and learn’ by talking to students and educators.

Reporters were not allowed to attend the student discussion with Schwinn and some Shelby County Schools leaders.

Douglass High entered Shelby County Schools’ turnaround program, known as the iZone, in 2016 and saw high academic growth in its first year. But test scores fell this past year as the state wrestled with online malfunctions.

Timmy Becton Jr., a senior at Douglass High, said he hopes for fewer tests and more projects to demonstrate what a student has learned. Those kind of assessments, he said, can help a student connect what they are learning to their daily life.

PHOTO: Shelby County Schools
Tennessee’s new education commissioner met with students at Douglass High School and Shelby County Schools leaders.

“We figured it would be a different way to measure and see how much knowledge a student really has on a specific subject,” he told Chalkbeat after meeting with Schwinn during a student roundtable session. “It’s a good alternative to taking tests.”

He said he was “surprised and happy” to see Schwinn actively seek student perspectives.

“I really think that’s the most important part because students are the ones going to school every day,” Becton said. “So, if you want to find a good perspective on how to solve a problem, it’s really great to talk to the people who are actively involved in it and the people who are actually experiencing these problems directly.”

The state’s annual testing window runs from April 15 to May 3.

School discipline

Michigan schools have expelled fewer students, but that may not be cause for celebration

PHOTO: Getty Images

Michigan schools have expelled far fewer students since the state enacted laws aimed at cutting back on expulsions. But an advocate who’s pushed for an end to zero-tolerance policies pointed out persistent problems and told elected state education leaders this week that, “We shouldn’t start celebrating yet.”

This is why: Peri Stone-Palmquist, executive director of the Ypsilanti-based Student Advocacy Center, told State Board of Education members that in the 18 months since the new laws took effect in 2017, expulsions have dropped 12 percent. But she’s concerned that too many school leaders don’t understand the law or are ignoring its requirements. And she believes some schools are finding other ways of kicking kids out of school without expelling them.

Michigan did away with zero-tolerance policies that had earned it a reputation for having some of the toughest disciplinary rules in the nation. In their place, lawmakers instituted new rules, such as requiring schools to consider seven factors — including a student’s age, disciplinary record, disability and seriousness of the incident — in making expulsion decisions.

“We have had districts and charters tell advocates that they would not consider the seven factors at all,” Stone-Palmquist said. Others aren’t sharing with parents and students how those seven factors were used. And she said there’s a general “lack of understanding of lesser interventions and the persistent belief that lengthy removals remain necessary.”

That’s a problem, she and others say, because of the negative consequences of kicking students out of school. Studies have shown that students kicked out of school are often missing out on an education and are more likely to get into trouble. Advocates also worry that expulsion exacerbates what they describe as a “school-to-prison” pipeline.

She said advocates are noticing that more students are receiving long suspensions, an indication that some schools are suspending students rather than expelling them. Hiding students in suspension data won’t work much longer, though. Michigan now requires schools to collect such data, which soon will be public.

Stone-Palmquist also said that some schools aren’t even going through the expulsion process, but simply referring students with discipline issues to “understaffed virtual settings.”

“Once again, the students who need the most get the least, and no one has to report it as an expulsion.”

Stone-Palmquist gave an example of a ninth-grader involved in a verbal altercation who was expelled for a long time for persistent disobedience, “despite our team lining up extensive community resources for him and despite the district never trying positive interventions with him.”

In another case, a fifth-grader was expelled for 180 days for spitting at another student who had done the same to them first. Stone-Palmquist said the seven factors weren’t considered.

“We were told at the appeal hearing that the student’s behaviors were too dangerous to consider lesser interventions.”

She and Kristin Totten, an education lawyer for the ACLU of Michigan, provided board members with statistics that some members found alarming. Totten noted that an ACLU review of data collected by the federal government shows that for every 100 students in Michigan, 38 days are lost due to suspension. In Oakland County, 26 days are lost for every 100 students. In Macomb County, it’s 35 days and in Wayne County, it’s 55 days.

One child who’s experienced trauma for years was repeatedly suspended from multiple schools. The 11-year-old has been diagnosed with post-traumatic stress disorder and attention deficit hyperactivity disorder. This school year, she’s been suspended for 94 days.

“Never once were the seven factors mentioned to her mother,” Totten said.

Stone-Palmquist asked board members to consider recommendations, including developing a model student code of conduct that incorporates the new rules, partnering with the advocacy center to request an attorney general’s opinion on what districts are required to do, and expanding data collection.

Tom McMillin, a member of the state board, asked whether the state should consider financial penalties, such as withholding some state aid.

“I’m a fierce advocate for local control. But in areas where the incentives might not be there to do what’s right … I’m fine with the state stepping in,” McMillin said.

Board member Pamela Pugh said she appreciated the push for the board to “move with great speed.” She said the data and stories provided are “compelling, as well as convincing.”

Stone-Palmquist said that despite her concerns, there have been some successes.

“Districts that used to automatically expel 180 days for fights, for instance, have partnered with us to dramatically reduce those removals with great outcomes,” she said. “We know alternatives are possible and that they actually help get to the root of the problem, prevent future wrongdoing and repair the harm.”

The Detroit school district didn’t come up during the hearing. But on the same day Stone-Palmquist presented to the state board, Detroit Superintendent Nikolai Vitti gave a presentation to his local board of education about what’s happened in the months since the district embarked on an effort to improve school culture by revising the student code of conduct, hiring deans for each school, and providing training on alternative discipline methods.

The bottom line: Vitti said that schools are booting out dramatically fewer students and greatly increasing alternative methods of discipline. In-school suspensions are up, given the push against out-of-school suspensions.

But the changes have also raised concerns. Some school staff have said the new rules are tying their hands. Vitti said it will take time for the changes to take hold, and he outlined some areas that need to improve, including more training.