Measuring schools

Beyond test scores: Indianapolis considers new ways to measure school quality

PHOTO: Alan Petersime

When Indiana’s school letter grades debuted in 2011, the purpose was to make it easier for the public to understand school quality — but key players in Indianapolis education say simpler is not always better.

Indianapolis Public Schools, in partnership with The Mind Trust and the mayor’s office, are developing “equity reports” that will use different kinds of school data to better inform parents and community members about the quality of roughly 100 schools in the city — both IPS schools and mayor-sponsored charter schools. The new reports will have much more information than the state-issued letter grades, which currently consider little more than student test scores.

“We know parents and families need more than A-F (grades),” said Kameelah Shaheed-Diallo, the senior vice president of strategy and community engagement with The Mind Trust. “This is a tool to give them something more.”

The new reports, expected to roll out in November, are modeled on similar reports used to measure schools in Washington, D.C. They come at time when states nationwide are searching for better ways to measure schools that don’t just rely on standardized tests.

Under the federal Every Student Succeeds Act, which will replace No Child Left Behind next year, states must include “non-academic” indicators in their accountability systems. The problem? Some of the new qualitative metrics are still being developed. They could include metrics such as graduation rates for high schools or things like school attendance or the results of student and staff surveys.

“This is very much a 1.0 moment for those measures,” said Thomas Dee, a professor from Stanford University’s graduate school of education. “I applaud efforts to pilot innovation, but I guess for me it’s not just about data. It’s about how it’s used, and my concern is that, if it just sits on a website, that it’s going to be under-utilized by parents and by school leaders and by taxpayers.”

The equity reports are one of two efforts underway in Indianapolis to develop a better school measurement systems — both for parents to make more informed school enrollment decisions and for schools to better reflect on what they need to improve. Equity reports are the parent- and community-facing piece.

Some of the data that will be used for the reports is already available from the Indiana Department of Education, but the reports will compile the information in an easy-to-find way that is more user-friendly, said Aleesia Johnson, the IPS’s innovation chief.

“The equity reports to me will be a way in which information is sort of packaged in a more transparent and clear way, both for (district) schools and public charter schools as well as for families,” Johnson said.

Both IPS and The Mind Trust are the process of meeting with community groups to find out exactly what parents want to know. Information on school leadership, discipline and attendance are already on the table.

IPS and the mayor’s office teamed up with The Mind Trust on this project after receiving a grant from the Michael and Susan Dell Foundation that is geared toward creating a unified school enrollment system. The reports will be folded into that system as an extra service for parents as they choose schools for their children.

That grant is also funding a second effort on school measurement that is designed to help schools internally assess how well they’re helping students.

The second effort is driven by UChicago Impact, a nonprofit group affiliated with the University of Chicago has been using school climate and culture surveys in Chicago schools for years. The Chicago model — currently being piloted in 20 IPS schools and 20 mayor-sponsored charter schools — is based on a theory about the “five essentials.” Those are five qualities that strong schools share, said Elliot Ransom, the director of the five essentials efforts at UChicago Impact.

According to the group’s research, if schools rate high on three of the five measures — effective leaders, collaborative teachers, involved families, supportive environment, ambitious instruction — they are 10 times more likely to see student improvement than schools that are weak in those areas. But just having the data isn’t enough if you don’t act on it, he said.

“Educators are awash in data at this point in time, and it’s sometimes hard to make sense of what we should pay attention to,” Ransom said “It was foolhardy to think that you give something to someone and they suddenly change. So part of that is making (the data) public because the conversation will fundamentally change about school quality.”

After seeing some progress in Chicago, the state of Illinois voted in 2013 to include the five essentials in tracking school quality. While Illinois’ accountability system doesn’t use explicit ratings like Indiana’s, Ransom said, the commitment from the state shows that Indiana, too, could use such information in a large-scale way.

Dee, the expert from Stanford, urges states to be open to new ideas and to experiment but warns educators against putting too much stock in qualitative measures. Surveys, in particular, can be tricky to use because people don’t always give honest answers.

“You would worry about low response rate, you would worry about the extent to which people filling out the surveys really understood what they were responding to,” Dee said. “So there’s concern there.”

For now, both the equity reports and the UChicago surveys are only funded through the end of this year. Whether they’ll continue next year will depend on whether parents and educators find them valuable and whether anyone is willing to pay for them in the future.

If the new measures are popular, they could serve as a model for the state education department as it expands Indiana’s new accountability system.

The state’s new A-F grading system does factor student test score improvement, graduation rates, Advanced Placement and International Baccalaureate courses and college readiness in a school’s grade, which means the system meets most of the new federal school accountability requirements for high school students. But state officials still have work to do to come up with metrics that go beyond test scores for elementary and middle schools.

Cynthia Roach, testing director for the Indiana State Board of Education, said the state is weighing its options, but no decisions have been made. Coming up with test alternatives and non-test based metrics is one of the goals of the new state panel that was created by lawmakers this year to explore a new accountability systems for Indiana schools after ISTEP is retired next year.

There haven’t been any conversations at the state level about how either of these new pilots might address gaps in Indiana’s accountability system, but the potential is there, said the Mind Trust’s Shaheed-Diallo.

“Changing the state accountability is a bigger lift than I think we have capacity for, but I think it gets to this demand question,” Shaheed-Diallo said. “If people know there are other ways to measure (schools), perhaps it does create additional momentum for those conversations.”

audit findings

Audit finds educational services lacking at Rikers Island, but corrections officials dispute report

PHOTO: Matt Green/Flickr

Corrections officials “systemically neglected” to ensure that young adult inmates knew they could enroll in school courses, according to an audit released Tuesday by Comptroller Scott Stringer. The audit also found that the city Department of Education failed to put mandated educational plans in place for incarcerated students with disabilities.

“That’s wrong, because if we’re going to reverse decades of backwards criminal justice policies, it’s going to be with bigger and better schools — not bigger and tougher prisons,” Stringer said in an emailed statement. “We have to do better.”

But officials from the city Department of Correction disputed the findings, and a response from the education department suggests the audit takes a narrow approach that misses “critical context.”

In 74 percent of sampled cases, the comptroller’s office couldn’t find evidence that inmates between the ages of 18 and 21 attended an orientation and were informed of their right to attend classes. In 68 percent of the sampled cases, auditors could not find required forms from inmates either accepting or rejecting educational services. In its response to the findings, a representative for the corrections department noted that some inmates may simply “refuse to sign the form.”

The corrections department wrote that it “disputes the overall finding” that inmates are not informed of their right to educational services. Furthermore, the audit “failed to capture” additional steps the department takes to do so.

In responses to the findings, included in the audit, corrections and education officials said all eligible students are offered the opportunity to attend classes. Every school day, the education department prints a list of eligible students who are in facilities with school programs, and the list is shared with corrections staff in the housing areas. Inmates who are interested can attend an information session and enroll immediately.

The corrections department’s response also states that inmates receive a handbook that includes information about enrolling in classes, and that signs are posted in common areas to inform inmates of their right to request educational services. Furthermore, the department conducts regular focus groups to create alternative programs of interest to young offenders who choose not to go to school, according to the response.

The audit also found that 48 percent of eligible students did not have a Special Education Plan, based on their Individualized Education Program, created for them within 30 days of beginning classes, as required. Those plans were never created for 36 percent of sample students, according to the audit.

The Department of Education responded that it is working to implement a new electronic system to track progress on education plans for students with disabilities, and that students who had such plans before being incarcerated continue to get the services they need.

The audit does note that all 16- and 17-year olds were receiving the educational services required by law. Those students have to attend school, whether they are incarcerated or not. Older students are eligible to receive educational services if they are under 21 years of age, have not already earned a high school diploma and will be incarcerated for 10 or more days.

Community voices

Memphians weigh in on Hopson’s investment plan for struggling schools

PHOTO: Marta W. Aldrich
Superintendent Dorsey Hopson speaks Monday night to about 175 educators, parents and students gathered to learn about Shelby County Schools' plan to make new investments in struggling schools

After years of closing struggling schools, Shelby County Schools is changing course and preparing to make investments in them, beginning with 19 schools that are challenged by academics, enrollment, aging buildings and intergenerational poverty.

This May, 11 of those schools will receive “treatment plans” tailored to their needs and based on learnings from the Innovation Zone, the district’s 5-year-old school turnaround initiative. The other eight schools already are part of a plan announced last fall to consolidate them into three new buildings.

Superintendent Dorsey Hopson and Chief of Schools Sharon Griffin talked up the new dynamic Monday night during a community meeting attended by about 175 educators, parents and students. In his proposed budget for next school year, Hopson has set aside $5.9 million to pay for supports for the 11 schools dubbed “critical focus” schools. 


Here’s the framework for the changes and which schools will be impacted.


Monday’s gathering was first in which Memphians got to publicly weigh in on the district’s new game plan. Here’s what several stakeholders had to say:

Quinterious Martin

Quinterious Martin, 10th-grader at Westwood High School:

“It really helped me to hear that the label of ‘critical’ is going to help us out, not pull us down. I was worried when I first heard our school would be on the list of critical schools, but I get it now. The point is to help the schools out, not make them feel worse. To me, one thing Westwood really needs is more classes to get us ready for our future careers, like welding or mechanics. My commitment tonight was to always improve in what I do.”

Deborah Calvin, a teacher at Springdale Elementary School:

“I enjoyed the presentation tonight. I think it’s so important to know everyone is on the same page. The plan will only be successful if everyone in the community is aware of what the goals are. I think they made it really clear tonight that just more money doesn’t help turn a school. It takes a lot of community support. We really need more parent involvement at Springdale. Children need support when they go home. They need someone to sit down with them and work through homework or read.”

Catherine Starks, parent at Trezevant High School:

“Honestly, I think this is just going through the motions and something to keep parents quiet. Some schools may be getting the supports they need, but not all of them are. Trezevant is one that is not. … We need good leadership and we need someone to be advocates for our kids. I want to see the kids at our school get the support they need from the principal, the guidance counselor, the superintendent. Trezevant has had negative everything, but now we need some positive attention. And we really need the community to step up.”

Neshellda Johnson and daughter Rhyan

Neshellda Johnson, fourth-grade teacher at Hawkins Mill Elementary School:

“Hawkins Mill has been in the bottom 5 percent for awhile and has been targeted (for takeover) by the state for about four consecutive years. …  It’s refreshing to see that, instead of putting us on the chopping block, the district is looking to actually invest in us and give us the tools we need so we can continue to have growth. … I’m looking to the district for academic supports with regards to reading, more teachers assistants, more time for teaching and less time for testing, and more after-school and summer enrichment programs. And in addition to supports for our students, I’m hopeful there will be supports offered for our parents. We have a need for mental health and counseling services in our area.”

You can view the district’s full presentation from Monday night below: