breaking news

Could new plans to adapt a test teachers love work for Indiana?

PHOTO: Alan Petersime

Indiana educators who want the state to consider replacing ISTEP with an exam that would give teachers immediate feedback about their students might have new reason to be optimistic.

Chalkbeat reported last week that teachers have called on the state to use something like the popular MAP test, which is beloved by teachers for providing real-time information about what students know. At the time, the company that makes the test said MAP isn’t designed to meet testing requirements demanded for federal accountability and Indiana law.

Now, however, the Oregon-based testing company that makes MAP says it’s looking for a way to respond to demand from states like Indiana that have school accountability systems that require a single yearly score to indicate which kids are performing at grade level.

“Our state assessment experts will work with states to understand their goals and devise full solutions to meet those needs,” Jason Mendenhall, the senior vice president of strategic solutions for the testing company, the Northwest Evaluation Association, wrote in an email.

The email said NWEA will soon be creating a new division to work specifically with states to create tests and accountability systems that both meet state and federal education guidelines and encompass aspects of MAP — Measure of Academic Progress — that teachers say they like.

The MAP, which is typically given three times a year, not only gives immediate results that can help guide instruction, teachers say, it also takes less time to administer than year-end comprehensive exams like ISTEP.

Plans to adapt to state accountability systems like Indiana’s had been in the works for some time, Mendenhall said in the email, but the company decided to announce it sooner than originally planned in response to questions raised by Indiana educators after Chalkbeat reported on the issue.

It’s not clear yet what the new NWEA tests will look like or how the company will manage to create a test that both meets Indiana testing and accountability requirements and works well as a teaching tool.

The NWEA initiative bears some resemblance to a vision for Indiana’s testing program that came up today during the second meeting of the state’s ISTEP replacement panel.

The committee, charged with choosing a new suite of tests for Indiana schools once ISTEP ends in 2017, convened at the state capitol Tuesday morning. Some educators and administrators spoke in favor of a “computer adaptive” testing system that measures how students improve over the course of a year, which is how MAP currently works.

“(Teachers) want a growth model, and they want it short and sweet,” said Roncalli High School Principal Chuck Weisenbach.

State Superintendent Glenda Ritz said she thinks new flexibility under the federal Every Student Succeeds Act would allow Indiana to move away from a pass/fail test and toward a MAP-like exam. Ritz said she thinks such a test could be modified to include parts that can measure students according to grade level, too.

“When the federal government gave the states, so to speak, the rights to use a computer adaptive test, I think they really did have in mind that you don’t have to have a pass/fail approach,” Ritz said.

Ritz’s vision, however, is likely to meet resistance from other members of the committee including House Education Committee Chairman Bob Behning and the panel’s chairwoman, Indianapolis Public School 93 Principal Nicole Fama.

Most members of the ISTEP panel were appointed by Gov. Mike Pence and the GOP leaders of the state legislature who have long supported high-stakes testing in schools.

Behning, Fama and other policymakers say they are more likely to support an exam that’s more similar to the current ISTEP in how it is administered once a year and measures where students perform within grade level expectations.

“At the end of the day, we have to come up with something that assesses (students) on grade level,” Fama said.

Behning also said he worried that using MAP-like tests that are given on computers could create issues for schools with fewer computers or less updated technology systems.

When the conversation ended Tuesday afternoon, there wasn’t a clear way forward. Parts of the panel’s conversations were very premature, some members said — It’s pointless to try to figure out test specifics before there’s a solid understanding what the test should be for.

At the group’s last meeting, discussion focused on the purpose of state exams. Experts said tests could be designed to determine whether students have improved from one year to the next, evaluate teachers or help rank schools and districts.

But they cautioned that a test that tries to do everything at once would be quite long.

Today, teachers and principals seemed against taking on too much with one test and instead urged the panel to remember that their charge involves more than coming up with a single new state test.

“I’m just concerned we’re getting (too much) into the details,” said Wendy Robinson, superintendent of Fort Wayne Schools. “Teachers don’t know any more what the state is expecting … We have a broken test that is being used like it’s a system, and it’s not.”

tar heel trivia

New education research? A good chance it’s from North Carolina.

PHOTO: Creative Commons/Boston Public Library

Barbeque. Basketball rivalries. The Blue Ridge Mountains.

Education research?

It’s something else North Carolina is known for, at least among a subset of social scientists.

“North Carolina has really done something special,” says Amy Ellen Schwartz, a professor and the editor of Education Finance and Policy, an academic journal.

“If you look over the last 20 years and focus on the highest quality work, it’s disproportionately work that comes from North Carolina data,” says Dan Goldhaber, an education professor at the University of Washington at Bothell.

North Carolina students aren’t more interesting or easier to find. But a disproportionate share of education research — and therefore, a disproportionate amount of what we know about how certain policies work — comes out of the Tar Heel State.

That’s because North Carolina has kept track of things like student test scores, teacher demographics, and school accountability data since the ‘90s, and also made that information more accessible to researchers than anywhere else.

It works well for those looking for data. But it also underscores a troubling reality: We know much less about how policies play out in places where data is hard to access — and in some cases, may be kept under lock and key for political reasons. That leaves the public to take the best lessons it can from a state that’s home to just 3 percent of the country’s public school students.

“The problem is that what you really want to do is look at lots of places,” said Schwartz, a professor at the Maxwell School at Syracuse University. “You want to be able to leverage the natural experiments and understand the variation in a way that’s really hard to do in one place.”

Of course, researchers in many cases do work productively with local officials to obtain data. And although it appears that North Carolina is the most commonly studied state in education policy, it is by no means the subject of the majority of academic papers. For instance, seven studies published in Education Finance and Policy over the last two years were focused on North Carolina — more than any other state or district, though over 30 others focused on K-12 schooling in the U.S used national data or data from elsewhere.

North Carolina’s popularity is tied to the fact that it is one of the few states where researchers can get student data (that has been anonymized) from a third party, in this case a research center established in 2000 that operates out of Duke University. In most states, the state education department or other state agency controls that information. Many states and districts lack the resources, streamlined systems, or staff capacity that North Carolina’s center has to meet researchers’ requests.

That center also separates policymakers and the keepers of the data — which may be crucial for ensuring information is made available.

“Not every place wants to open up their data and say, ‘Study what you want,’” said Schwartz. “The risk is that a researcher investigates something or casts it in a way that’s not positive for the school district.”

Goldhaber echoed this. “If you’re talking to somebody who’s involved with politics … they’re going to see everything through a political lens. And that when it comes to evaluating programs and policies, people often don’t see much upside,” he said.

In North Carolina, local researchers realized the importance of tracking students and schools over time, according to Duke’s Clara Muschkin, the faculty director of the data center.

When Goldhaber was studying schools there in the 1990s, he recalled, “There was a real belief that people ought to study these issues, and that was kind of pervasive under Gov. Jim Hunt.”

That extended to research that Hunt’s administration might not like. For instance, Goldhaber was interested in studying whether teachers who attained National Board certification were more effective in the classroom. Hunt was the founding board chair of the organization that awarded those certifications, and Goldhaber’s research had previously shown that certification types didn’t make much difference. But that didn’t stop the administration from providing that data to Goldhaber, who ultimately found North Carolina’s board certified teachers were particularly effective.

It’s impossible to say how often political concerns play a role in keeping data from researchers. When politics is involved, researchers themselves may not know, and if they do, they may not want to publicize it in hopes of eventually working out an agreement. (This reporter has heard frequent complaints about politics getting in the way of data access — but in most cases those are made off the record.)

A more subtle method of interference is when officials decide not to collect data in the first place that researchers might use to reach unflattering conclusions. California, Goldhaber said, is a particular culprit.

The largest state in the country has weakened, or declined to improve, its data systems since 2010, and the information that exists is not readily available to researchers. Governor Jerry Brown has argued that educational data is of little use to teachers and schools, and feeds into a test-focused mentality of schooling.

“You are not collecting data or devising standards for operating machines or establishing a credit score,” wrote Brown in a critique of the Obama administration’s Race to the Top program, which encouraged more data collection. “I sense a pervasive technocratic bias and an uncritical faith in the power of social science.”

Goldhaber has found it difficult to study the state’s education policies.

“There is just basic data that we could not get out of California,” he said, referring to a study he and colleagues are undertaking there.

Some places are becoming more cognizant of concerns about a lack of quality research about their schools. In Washington, D.C., the city council is considering funding an education research group and may make its data widely available to researchers. In California, some advocates and policymakers have pushed for improving its data systems, an idea the state’s likely next governor has backed.

In the meantime, those interested in key education questions — in California, DC, and elsewhere — can always look to North Carolina for answers. That’s largely a good thing, says Goldhaber.

“The fact that we are learning things in North Carolina is tremendously useful for informing policy and practice in other states,” he said.

right to know

A bastion of student data privacy, Colorado yields a bit to demands for more openness

PHOTO: Getty Images

Colorado education officials are reconsidering the data privacy rules that for three years in a row have hidden large amounts of student achievement data from public view.

With Thursday’s release of state test results, the public has greater ability to see how well certain groups of students perform on state tests compared with their peers than they’ve had since 2015, when the state adopted a much more stringent approach.

The main change this year is that the public can see results separated by race and ethnicity, disability status, English language learner status, and economic status at the school level, rather than only by grade levels within schools. Combining results across grades creates larger sample sizes, so that in some cases they no longer had to be redacted. The practice of obscuring results for small groups of students had led to large amounts of missing information and vexed advocates for school improvement.

And state education officials are considering additional changes for 2019 that should make more information available – though just how much remains to be seen.

“In this latest release, we were very pleased to be able to see how kids are making progress,” said Van Schoales, executive director of A Plus Colorado, an education reform advocacy group that focuses on research. “We can now see if they met or exceeded standards for groups of 16 or more. We thank the Colorado Department of Education for figuring out some methods for showing that.”

A Plus Colorado was part of a coalition that wrote the Colorado Department of Education last November calling for more transparency. That group includes the Colorado Children’s Campaign, the business-oriented education reform group Colorado Succeeds, civil rights groups like Together Colorado and Padres y Jóvenes Unidos, Democrats for Education Reform, the Colorado League of Charter Schools, and the Colorado Freedom of Information Coalition.

“The current lack of transparency in student data and results is a major challenge to our collective efforts to build a better education system,” they wrote. “Results that are not transparent and comprehensive not only undermine our ability to improve students’ education, but call into question the honesty and integrity of the entire accountability system.”

Advocates and state education officials met through the spring and summer to see if any common ground could be found. The state also hired an outside firm, HumRRO, to assess its data practices.

That firm, according to Deputy Commissioner Alyssa Pearson, found that Colorado was right to change its practices in 2015 but that there was room for greater transparency while still protecting student privacy.

This debate doesn’t involve test results of individual students but rather information about how groups of students at particular schools perform. Advocates for school accountability say the ability to know, for example, how well students with disabilities or those learning English perform in comparison with their peers is key to knowing how well schools are serving all their students.

Colorado was previously one of the most open states in providing detailed information about student performance. But starting in 2015, the Colorado Department of Education withheld a lot more information, particularly when results were separated out by categories of students. Instead of hiding results just for groups of 16 students or smaller, the state hides results from additional categories of students to prevent a careful observer from working backward to determine what was in the redacted field. This is known as complementary suppression.

And instead of showing how many students scored in each tier on state tests, Colorado shows what percentage of students met or exceeded expectations. That makes it hard to know if students who didn’t meet expectations were just below the threshold or way at the bottom.

Colorado plans to keep obscuring results from small groups of students and to continue the practice of complementary suppression. There are still missing results in the 2018 data publicly released so far. However, by combining results from different grade levels within schools, a more complete picture emerges of how each subgroup of students is being served.

No single state or federal law dictates the specific practices that Colorado has adopted. In fact, federal law requires that states make disaggregated data about student performance available to the public, so long as it doesn’t end up revealing personally identifiable information. These practices are what state officials decided was necessary to prevent the release of information traceable to individual students. That provides some wiggle room to reassess those practices.

Pearson said state officials are focused on who needs to know what information and for what purposes and then trying to accommodate that within privacy requirements. They’re also revisiting the question of how likely it is a “reasonable person” could trace certain pieces of data back to individual students.

In addition to making detailed school-level data available, Colorado has revamped its own online search tool, Data Lab, to make it easier for the public to search for how different groups of students performed at particular schools. Right now the information is available across multiple spreadsheets available for download. The data from 2018 tests should be incorporated into Data Lab sometime in September.

Another change on the table: Reducing the threshold for suppressing data. State officials will decide by the end of the year about whether to lower the bar for concealing group data to 10 students, from the current 16. That would mean less data would be suppressed – but how much less isn’t clear yet.

Pearson said the state hopes to make a final decision by the end of the year so that changes can be incorporated into the 2019 testing cycle. It hasn’t been decided yet whether administrators would make that decision on their own or whether the State Board of Education would vote on the policy change.

Along with pressing the state to release more information, advocates for more transparency have also launched a Right to Know website and campaign to encourage districts and schools to share more information with families. Districts are required by law to share information from state assessments with parents, but practices vary widely across the state.

At a recent meeting of the State Board of Education, members of the Colorado Youth Congress, part of the Right to Know coalition, urged the state to do more to put this information in the hands of the people most affected.

“As a student, I want to know if my school is performing strongly or not so I can understand the setting I’m in and if I want to change to a different setting that would benefit me more,” said Jasmine Kabiri, a junior at Silver Creek High School in Longmont. “In a different situation, if we’re shown the performance data of my school, I could help motivate my school toward advancing themselves.”