test cred (updated)

Calls for investigation into test credibility go unanswered

State Board of Regents Chancellor Merryl Tisch is calling for state exams to be more “defensible,” but a study investigating test score credibility requested a year ago by the state’s testing oversight board has still not received a go-ahead.

The committee first formally asked the state education department to join an academic study on the state tests in the fall of 2008, said chair Howard Everson. The education department declined but did not rule out future participation. Since then, Everson has received no requests to revisit the idea, he said in an interview yesterday.

“It’s hard to trust the data right now,” said Everson, a psychometrician who is also a senior fellow at the City University of New York. Everson’s committee, the state Technical Advisory Group, is charged with monitoring the state testing process.

The study, which Everson is developing with Harvard education professor Daniel Koretz, would investigate a phenomenon called score inflation. Score inflation happens when rises in test scores reflect something other than actual learning — for instance, bending the test rules (e.g., giving students more time) or even cheating. Tests may also become so predictable that teachers learn to coach students on how to ace them, experts say.

The researchers propose measuring what test scores really represent by creating a “test within a test” that would assess the same skills but in a variety of formats. If students score worse on the new test than on the regular test, Everson said, that would suggest that score inflation is at work.

The key is to give both tests to the same group of students at around the same time. This would eliminate variables that may account for existing differences in results between state exams and alternative assessments such as NAEP.

If such a study happened, Everson said, it would be the first time that New York state accountability tests have ever been rigorously examined for possible score inflation.

Requests for comment by a state education department spokesman on whether a study on score inflation is being considered were not returned.

The research study designed by Everson and Koretz is still in its preliminary stages, Everson said, and the researchers are in early discussions with several states to participate.

New York would be an excellent site for research because of the high stakes attached to the exams, Koretz said in an interview earlier this year.

“In the old days, when the pressure to raise scores wasn’t so high, it mattered less that tests were somewhat predictable,” he said. “But when the pressure is as high as it is now – everywhere, but particularly in New York with value-added measures for teachers and so on – people have every incentive to look for predictable patterns and to narrow their instruction to focus on those patterns.”

Columbia University sociology researcher Jennifer Jennings has found that almost identical questions have appeared on each state math exam since 2006, making it easier for schools to teach to the test.

Koretz said that with increased attention to high-stakes testing in New York, he hopes that the pressure for a thorough look into score inflation will mount.

“I hope that New York can be what finally breaks the ice,” he said.

Everson was quoted in the Times as saying that the state tests are “about as good as we can build them.” In an interview with GothamSchools, Everson elaborated on that appraisal, saying that the technical operations of state testing–how the tests are administered and scored–are strong. But he made a distinction between the way the tests are being given and the way the results are being interpreted.

“It’s just a hypothesis, but we worry that there would be more score inflation,” Everson said. “We think there would be more teaching to the test.” But there is no way to be sure without more rigorous examination, he said.

UPDATE: The New York State Department of Education is currently researching the possibility of using an “audit mechanism,” like the “test within a test” that Everson described, to guard against score inflation in their standardized tests, department spokesman Jonathan Burman said.

Newark Enrolls

After changes and challenges, Friday’s deadline to enroll in Newark schools finally arrives

PHOTO: Patrick Wall/Chalkbeat
A student fills out an information sheet at Central High School's booth at the citywide school fair in December.

Newark families have just a few hours left to apply to more than 70 public schools for next fall.

At noon on Friday, the online portal that allows families to apply to most traditional and charter school will close. After that, they will have to visit the district’s enrollment center. Last year, nearly 13,000 applications were submitted.

The stakes — and stress — are greatest for students entering high school. Each year, hundreds of eighth-graders compete for spots at the city’s selective “magnet” high schools, which many students consider their best options.

This year, those eighth-graders have to jump through an extra hoop — a new admissions test the magnets will use as they rank applicants. District students will sit for the test Friday, while students in charter and private schools will take it Saturday.

That’s news to many parents, including Marie Rosario, whose son, Tamir, is an eighth-grader at Park Elementary School in the North Ward.

“I don’t know nothing about it,” she said. District officials have been tight-lipped about what’s on the new test, how it will factor into admissions decisions, or even why introducing it was deemed necessary.

Students can apply to as many as eight schools. Tamir’s top choice was Science Park, one of the most sought-after magnet schools. Last year, just 29 percent of eighth-graders who ranked it first on their applications got seats.

“I’m going to cross my fingers,” Rosario said.

Students will find out in April where they were matched. Last year, 84 percent of families applying to kindergarten got their first choice. Applicants for ninth grade were less fortunate: Only 41 percent of them got their top choice, the result of so many students vying for magnet schools.

This is the sixth year that families have used the online application system, called Newark Enrolls, to pick schools. Newark is one of the few cities in the country to use a single application for most charter and district schools. Still, several charter schools do not participate in the system, nor do the vocational high schools run by Essex County.

Today, surveys show that most families who use the enrollment system like it. However, its rollout was marred by technical glitches and suspicions that it was designed to funnel students into charter schools, which educate about one in three Newark students. Some charter critics hoped the district’s newly empowered school board would abolish the system. Instead, Superintendent Roger León convinced the board to keep it for now, arguing it simplifies the application process for families.

Managing that process has posed challenges for León, who began as schools chief in July.

First, he ousted but did not replace the district’s enrollment chief. Then, he clashed with charter school leaders over changes to Newark Enrolls, leading them to accelerate planning for an alternative system, although that never materialized. Next, the district fell behind schedule in printing an enrollment guidebook for families.

Later, the district announced the new magnet-school admissions test but then had to delay its rollout as León’s team worked to create the test from scratch with help from principals, raising questions from testing experts about its validity. Magnet school leaders, like families, have said they are in the dark about how heavily the new test will be weighted compared to the other criteria, including grades and state test scores, that magnet schools already use to rank applicants.

Meanwhile, León has repeatedly dropped hints about new “academies” opening inside the district’s traditional high schools in the fall to help those schools compete with the magnets. However, the district has yet to hold any formal informational sessions for families about the academies or provide details about them on the district website or in the enrollment guidebook. As a result, any such academies are unlikely to give the traditional schools much of an enrollment boost this year.

District spokeswoman Tracy Munford did not respond to a request Thursday to speak with an official about this year’s enrollment process.

Beyond those hiccups, the enrollment process has mostly gone according to plan. After activating the application website in December, the district held a well-attended school fair where families picked up school pamphlets and chatted with representatives. Individual elementary schools, such as Oliver Street School in the East Ward, have also invited high school principals to come and tell students about their offerings.

American History High School Principal Jason Denard said he made several outings to pitch his magnet school to prospective students. He also invited middle-school groups to tour his school, and ordered glossy school postcards. Now, along with students and families across the city, all he can do is wait.

“I’m excited to see the results of our recruitment efforts,” he said. “Not much else is in my control — but recruitment is.”

reunion

Jubilation, and some confusion, as Denver schools begin their post-strike recovery

PHOTO: Eric Gorski/Chalkbeat
Lupe Lopez-Montoya greets students at Columbian Elementary School on Thursday after Denver's three-day teacher strike.

Lupe Lopez-Montoya got the text message at 6:15 a.m. The strike was over, the colleague who’d been keeping her up to speed on news from her union told her.

And so as district and teachers union officials celebrated their deal at the Denver Public Library, Lopez-Montoya on Thursday resurrected her regular commute to Columbia Elementary School, the northwest Denver school where she’s the longest-serving tenured teacher.

For the past three days, Lopez-Montoya had stayed out of school as Denver Public Schools and the Denver Classroom Teachers Association sparred over how teachers in Denver are paid. She still had unanswered questions about the future. But first, it was time to greet students.

One girl ran across the sidewalk to give a hug. “Ms. Montoya’s back,” she shouted, burying her face in the teacher’s side.

Lopez-Montoya began to welcome students into the building. “Line up and you can go get breakfast,” she told one boy. “I’m happy you’re here today.”

The moment kicked off what was a not-quite-normal day in Denver schools. With a deal coming just an hour before some schools were due to start for the day, teachers and families had little time to adjust their plans. The district also had too little time to reopen early childhood classes that have been closed all week; those will reopen on Friday, officials said.

Building principals got an email around 7 a.m. telling them that the strike was over and that many teachers would be coming back to work. In some buildings, central office staff and substitutes who had been filling in for teachers were already on site when teachers returned to work. And some students who commute long distances didn’t get the word in time to go to schools.

By early afternoon, about 81 percent of teachers in district-run schools had shown up to work, and about 83 percent of students had, according to Denver school district spokesman Will Jones.

In the parking lot outside Mathematics and Science Leadership Academy, a west Denver elementary school, teachers still clad in the red of their cause gathered in the parking lot to walk in together. Asked what the mood was, reading intervention teacher Denise Saiz said, “Relief!” and her colleagues responded with an enthusiastic “Yes!”

“We’re really excited to get back to the kids,” said fifth grade teacher Emily Mimovich.

While strike participation was high at the school — it was founded with support from the state and local union as a teacher-led school — the teachers said they did not believe there would be hard feelings between those who walked and those who didn’t.

“We have such a good relationship,” said Reyes Navarro, who teaches Spanish to kindergarten and first-grade students. “We understand that you have to do whatever is right for you.”

Parents also said they were eager to have their children get back to school.

Sarah Murphy, who has a third-grader and a 4-year-old preschool student, said this morning was a “bit of a scramble, albeit a welcome one.” She had both children signed up for a day camp at the University of Denver and was unsure if she should send them to school or to camp because she wasn’t sure teachers would be back at school. Then she got word that camp was canceled for both children — but Denver Public Schools preschools are still closed.

“We were left trying to figure out what to do with our ECE4 since they were still closed,” she wrote to Chalkbeat. “We know how hard this has been on everyone, but this morning proved that the ECE program was the hardest hit, short notice every step of the way. We very much look forward to tomorrow when both are back in their normal school routines.”

Not everyone returned to school. Some teachers said they needed a day to collect themselves after an emotional experience. Judy Kelley, a visual arts teacher at Martin Luther King Jr. Early College, said she and many of her colleagues live too far away from the school to get there on short notice.

But the majority of teachers who went back to work Thursday describe positive experiences returning to their schools. Ryan Marini, a social studies teacher and football coach at South High School, called it the “best day” in 17 years of teaching.

Suzanne Hernandez, a first-grade teacher at Westerly Creek Elementary in Stapleton, said she and other teachers gathered to walk into school together at 7:45 a.m.

Teachers thanked the subs from central office who were in the building. They headed back to their roles in central offices.

Early in the morning, Hernandez sent a message to her students’ parents to let them know teachers would be back in school. The reaction from students as she greeted them was “overwhelming,” she said.

With the help of paraprofessionals who had prepared lessons, Hernandez said teachers were able to jump right back to where they left off last week.

As far as Valentine’s Day, that celebration will be pushed back until Friday afternoon, she said. For now, teachers are having their own celebration, happy to be back in the classroom.

“I think if there’s any sort of effect from the strike, it’s been a positive one,” she said. “It’s been a very unifying experience.”

Allison Hicks, a teacher at Colfax Elementary, said returning teachers were greeted with hugs, music, smiles and tears.

“After finding out an hour ago we were going back to work, I scrambled home to get ready,” she wrote to Chalkbeat. “I am exhausted, but so excited to be back with my students. This day won’t be normal, but knowing I did my part to put my students first is the best feeling to have.”

Kade Orlandini, who teaches at John F. Kennedy High School, said most of the teachers are back in her building, despite the short notice.

“Teachers were ready, and we are jumping right back into instruction,” she wrote. “Attendance seems lower than normal, but students who are here are ready to learn. The atmosphere in the school is very positive all around.”

What was your experience going back to school? What other questions do you have? Take our survey

Chalkbeat’s Ann Schimke, Erica Meltzer, and Yesenia Robles contributed reporting to this article.