Are Children Learning

Test format questions hold up state board vote on ISTEP passing scores

PHOTO: Scott Elliott
The state board decided not to vote on projected ISTEP passing scores this month.

Last-minute questions about differences in the 2015 online and paper-pencil ISTEP tests postponed an Indiana State Board of Education vote to set passing scores on the exams today.

The delay put off a discussion about big drops expected in the number of students who passed ISTEP last year if the board adopts the recommended passing scores. The passing rate declines are connected to tougher new academic standards, which made for more difficult ISTEP tests.

Board members, who often have been at odds with state Superintendent Glenda Ritz and the Indiana Department of Education staff, quickly complained that the latest delay was another example of the Ritz’s team failing to provide timely information to the board.

This delay is the second for ISTEP scores after the testing company that creates the exams, California-based CTB, reported problems with short-answer questions back in August.

“In my view this was avoidable,” board member Gordon Hendry said. “We could’ve voted on cut scores today if the information was delivered like it should have been.”

But Danielle Shockey, deputy state superintendent, said nothing was withheld from the board

A draft study doesn’t actually need to be done first, she said. The process used to come up with the recommended passing scores could still be approved, and technical adjustments could come later.

“It is the same valid work we’ve done previously for other tests,” Shockey said. “There is absolutely no reason to pause or delay the test cut-score setting today.”

ISTEP scores factor into many critical statewide decisions, such as school A-to-F grades and teacher evaluations, as well as teacher pay decisions. At last month’s board meeting, the Indiana Department of Education said school grades wouldn’t be released until January 2016.

The comparison study is very technical. It details statistical tests performed on test scores and questions from both the online and paper-pencil tests. The findings seem to suggest that some questions on certain tests are harder on the online test than the paper-pencil test and vice versa, particularly in the math section.

“The concerns center on a report by the Department of Education from October 2, but was not provided to the test’s Technical Advisory Committee, a team of state and national experts hired to monitor the testing process, and the State Board of Education until Tuesday night,” said a statement from the state board. “That report raises serious questions about potential differences in difficulty between the online and paper/pencil tests.”

Comparison studies are not unusual, in Indiana or in general. They are done to see whether different test formats affect the validity of the test scores.

But the state board wants its own test experts to review the study, and it says the department didn’t fulfill its end of the bargain. Board member Sarah O’Brien said she requested this study back in July and was told by the education department that it would be finished before work on passing scores started.

“In order for us to justify such a huge decision for the field, we need to be able to say we have this unbiased information saying we are on-track,” said O’Brien, a teacher from Avon. “I don’t feel we have that.”

Karla Egan, a test expert on the technical advisory committee that helps oversee the score-setting process, said the comparison study doesn’t necessarily show something is wrong. But the test experts should have it before they approve recommended cut scores.

“We don’t know that there’s a problem,” Egan said. “We just need more time.”

State board test director Cynthia Roach said comparison studies should, and typically have been, completed before cut-off scores are created to begin with. When asked why the process continued without the study, state board spokesman Marc Lotter said they were assured the tests were comparable by a CTB representative, but no actual data was given.

Michelle Walker, the department’s test director, said based on the draft study by CTB, she’s confident the paper-pencil and online tests are comparable and will prove valid. The state board’s review of the process doesn’t have to hold things up.

“The draft came to us on (Oct. 2) so we could discuss it with CTB last week, which we did do,” Walker said. “If they want feedback … I think that’s fine. I think that’s a third-party, secondary piece in the process.”

The board was expected to vote today on ISTEP passing scores, which projections said could drop by about 16 percentage points in English and 24 percentage points in math compared to 2014. The drops are worse than in prior years, as ISTEP scores have rarely swung up or down by more than a few percentage points since 2010.

Part of that difference is because the 2015 ISTEP test was written to measure new, more challenging academic standards. For that reason, lower scores were expected, and some educators and advocates say other states that have switched standards saw similar changes.

Ritz said it’s possible the board could vote on the recommended passing scores in two weeks when it holds a special meeting to discuss new proposed high school diplomas on Oct. 28.

focusing in

Black student excellence: Denver school board directs district to better serve black students

PHOTO: Helen H. Richardson/The Denver Post
Mary Getachew, 15, right, laughs with her peer mentor Sabrin Mohamed,18, left, at Denver's North High School in 2016.

Every Denver public school soon will be required to develop a plan to boost the success of black and African-American students by embracing their strengths rather than focusing on the challenges they face.

That’s according to a resolution unanimously passed Thursday night by the Denver school board. The resolution, which would also require district employees to take training on implicit bias, was shepherded by Jennifer Bacon, who was elected in 2017 to represent northeast Denver and is one of two black members on the diverse school board. Longer-serving board members said it was overdue.

“With good intentions, we were battling the idea that singling out a group of students was not acceptable,” said Happy Haynes, who has served on the board since 2011. “We were always talking about, ‘all students, all students.’”

In doing so, Haynes said, “we lost sight of so many of our students. So I really celebrate this change in our thinking.”

Denver Public Schools’ data show big disparities in how black students are served by the district. While 13 percent of the approximately 93,000 students are black,

  • 28 percent of out-of-school suspensions last year were given to black students,
  • 16.5 percent of students identified as having a disability were black,
  • Just 10 percent of students enrolled in rigorous high school courses were black.

The focus on black students comes after more than a year of relentless and high-profile advocacy from black parents and activists, and 2½ years after a damning report about how black teachers and students are treated in Denver Public Schools.

Known colloquially as the Bailey Report, it was based on interviews with black educators conducted by former school board member Sharon Bailey, who has studied racial dynamics in Denver. It found that black educators feel isolated and mistreated by the district, and perceive that black students are more harshly disciplined in part because the young white women who make up a sizeable portion of the teacher workforce are afraid of them.

The report led to a task force, which presented the district with 11 recommendations. Among them: offering signing bonuses to help attract more black teachers, making student discipline data count toward school ratings, and requiring each school to create a plan “designed to strengthen relationships between African-Americans and schools.”

Nearly two years later, none of that has happened. And much of what the district has done has been voluntary for teachers and schools. Meanwhile, the data keeps mounting.

Last year, 67 percent of black students graduated on time, meaning within four years of starting high school, compared with 78 percent of white students. On state math tests, 17 percent of black students in grades three through eight scored on grade level, compared with 65 percent of white students. The literacy gap was similar.

Avery Williams, a senior at George Washington High School, told the school board at a work session in December that “there’s an awkwardness around being black” in Denver schools.

“Teachers, specifically white teachers, don’t know how to act around me,” Williams said. Many of her classmates, she said, “do not know how to have respectful conversations because they’re afraid of being offensive or because they’re not educated in the right terminology.”

Michael Filmore, a junior at East High School, spoke about being one of only a few black students in his more rigorous classes, an experience Williams shares. After taking remedial classes his freshman year at East, Filmore said he decided to take all honors classes as a sophomore. He also took the public bus to school and was often late for first period.

“I would walk in the classroom and I would feel like I didn’t belong there,” Filmore told the board. “I felt uncomfortable and that I shouldn’t be in these classes. I was pressured. I eventually dropped the class. My junior year, I felt that I would never let myself down again.”

At that December session, Bacon expressed a desire to more explicitly address issues affecting black students. The district has put that kind of focus on students learning English as a second language, many of whom are Hispanic, after a federal judge found the district was violating their rights. Under that order, the district has developed specific methods for teaching English language learners. It requires all new teachers to get certified to teach them.

Bacon and others questioned why that hasn’t happened for black students, as well.

“It’s not because there’s a lack of effort, will, or love,” Bacon said in an interview. “I think it’s because we’re not organized properly and we don’t have an internal stake in the ground around expectations, outcomes, and accountability measures. People want to see DPS is doing that.”

Her fellow board members agreed. On Thursday, they took turns thanking her for bringing forth the resolution, which directs the district to do several things:

  • Require all schools, including district-run and charter schools, to review data about student academic performance, discipline, and referrals for special education to understand how each school’s black students are doing “on an individual level”
  • Require all schools to set goals for supporting black students that prioritize giving them “access to grade-level and more rigorous coursework”
  • Require school leaders to articulate how they will monitor progress toward their goals
  • Train all district staff on implicit bias and culturally responsive education
  • Conduct an “equity audit” to understand what the district is doing well and what it is not to figure out how it “can better prioritize the success of our black students”

It will now be up to new Superintendent Susana Cordova, who made equity a cornerstone of her bid for the district’s top job, to carry out the directive. The resolution gives her until May 31 to come up with a plan that would go into effect by the start of the next school year.

“We know that we have a painful and inequitable history of outcomes for our students,” Cordova said. “But facing this with courage, facing this in community, facing this with our stakeholders, our parents, our family members, our community members, and our students holding us accountable, I believe deeply in the ability of people to come together to solve these problems.”

making the rounds

Tennessee’s new education chief ‘very confident’ that online testing will be smooth in April

PHOTO: Shelby County Schools
Tennessee's new education commissioner Penny Schwinn (second from left) met with Douglass High School students and Shelby County Schools leaders Friday.

As Tennessee’s new education commissioner wrapped up her second week on the job by visiting four schools in Shelby County, Penny Schwinn said she feels “very confident” the state has learned from its mistakes in online testing.

During the more than three-hour ride to Memphis on Friday, Schwinn said she continued to pore over documents showing evidence that the corrections the state department staff have put in place will work.

“I feel very confident that our team has looked into that,” she told reporters in a press conference after meeting with students. “They’re working with the vendor to ensure that testing is as smooth as possible this year.” Currently the state is working with Questar, who administered TNReady online last year.

She also said the state’s request for proposals from testing vendors, which is already months behind, will be released in about two weeks.

PHOTO: Shelby County Schools
From left: John Bush, principal of Douglass High School; Penny Schwinn, Tennessee Education Commissioner; and Joris Ray, interim superintendent for Shelby County Schools.

“No later than that,” she said. “We hope and expect to have a vendor in place before the end of the fiscal year,” in late June.

The day Schwinn was hired, she said getting state testing right would be her first priority. Three years of major technical failures have severely damaged the trust educators and parents have in the state’s test, TNReady. It is the main measure of how schools and teachers are doing, but state lawmakers exempted districts from most testing consequences in 2018.

From Schwinn’s first day on the job: Tennessee’s new education chief wants to ‘listen and learn’ with school visits

Prior to talking with reporters, Schwinn said she heard “hard-hitting questions” from several students at Douglass High School in Memphis about what the state can do to improve education. Schwinn has said she will visit Tennessee schools throughout her tenure to ‘listen and learn’ by talking to students and educators.

Reporters were not allowed to attend the student discussion with Schwinn and some Shelby County Schools leaders.

Douglass High entered Shelby County Schools’ turnaround program, known as the iZone, in 2016 and saw high academic growth in its first year. But test scores fell this past year as the state wrestled with online malfunctions.

Timmy Becton Jr., a senior at Douglass High, said he hopes for fewer tests and more projects to demonstrate what a student has learned. Those kind of assessments, he said, can help a student connect what they are learning to their daily life.

PHOTO: Shelby County Schools
Tennessee’s new education commissioner met with students at Douglass High School and Shelby County Schools leaders.

“We figured it would be a different way to measure and see how much knowledge a student really has on a specific subject,” he told Chalkbeat after meeting with Schwinn during a student roundtable session. “It’s a good alternative to taking tests.”

He said he was “surprised and happy” to see Schwinn actively seek student perspectives.

“I really think that’s the most important part because students are the ones going to school every day,” Becton said. “So, if you want to find a good perspective on how to solve a problem, it’s really great to talk to the people who are actively involved in it and the people who are actually experiencing these problems directly.”

The state’s annual testing window runs from April 15 to May 3.