guesswork

Test analyst: Reading exam bar even lower than critics say

Passing the state reading test might be even easier than recent criticism has suggested, a former Department of Education testing analyst is arguing.

Independent statistician Frederick Smith examined the way free-response questions were graded and found that virtually every student received enough points on that section to then pass the test by guessing randomly on the multiple-choice questions.

Smith’s finding expands on an informal study by a city teacher who concluded that arbitrarily filling in a pattern of multiple-choice answers and leaving the open-response section blank could yield scores high enough to promote a student to the next grade in New York City. That amounts to eight correct multiple choice answers on the fifth-grade English Language Arts exam.

Smith found that students who answered only 6 multiple choice questions correctly almost certainly would also pass the Level 2 bar. That’s because an overwhelming majority–99 percent–of fifth-graders who took the exam in 2009 received at least two points on the open-ended essay and free response sections, which would boost them to the eight-point cut-off level.

In addition, more than 97 percent of students received three or more points on that section, meaning that they would have had to answer just five multiple-choice questions correctly to receive a Level 2 score.

“There’s no point in setting a cut-off score that you can guess your way through,” Smith said. By effectively reducing the number of correct multiple choice answers needed, he said, the tests significantly increased the odds that a student could guess randomly and still hop the Level 2 bar.

Columbia University Teacher’s College professor James Corter calculated the likelihood that randomly guessing students would score at these levels. He found that more than 57 percent of students guessing would answer at least six answers correctly. Just over 75 percent of guessing students were likely to receive five or more correct answers.

The statistical calculations apply only to students who are truly guessing randomly, but could apply to a student with no knowledge of test content.

The number of students receiving failing Level 1 scores has declined dramatically over the past four years. On the ELA exams Smith analyzed, the number of students scoring at Level 1 declined by 95 percent between 2006 and 2009.

Corter cautioned that even though almost all students are receiving at least partial credit on the free-response questions, it doesn’t necessarily mean the questions are easy.

But Smith said the low scoring bar harms the tests’ credibility.

“The fact remains that this is the first year you can waltz your way to a level 2,” Smith said. The scoring effectively removes the failing Level 1 as a meaningful tier in the test evaluations, he said.

Smith has been publicly calling for an independent testing ombudsman in New York City since the early 1980s. He said the results of his examination highlight the need for a truly independent auditor of the state testing program.

“It has to have real authority, real teeth, and be able to get data without having to beg for it from the Department of Education,” he said. “This is the era of accountability. It’s time for some real answers here.”

money matters

Report: Trump education budget would create a Race to the Top for school choice

PHOTO: Official White House Photo by Shealah Craighead
President Donald Trump and U.S. Secretary of Education Betsy DeVos participate in a tour of Saint Andrews Catholic in Orlando, Florida.

The Trump administration appears to be going ahead with a $1 billion effort to push districts to allow school choice, according to a report in the Washington Post.

The newspaper obtained what appears to be an advance version of the administration’s education budget, set for release May 23. The budget documents reflect more than $10 billion in cuts, many of which were included in the budget proposal that came out in March, according to the Post’s report. They include cuts to after-school programs for poor students, teacher training, and more:

… a $15 million program that provides child care for low-income parents in college; a $27 million arts education program; two programs targeting Alaska Native and Native Hawaiian students, totaling $65 million; two international education and foreign language programs, $72 million; a $12 million program for gifted students; and $12 million for Special Olympics education programs.

Other programs would not be eliminated entirely, but would be cut significantly. Those include grants to states for career and technical education, which would lose $168 million, down 15 percent compared to current funding; adult basic literacy instruction, which would lose $96 million (down 16 percent); and Promise Neighborhoods, an Obama-era initiative meant to build networks of support for children in needy communities, which would lose $13 million (down 18 percent).

The documents also shed some light on how the administration plans to encourage school choice. The March proposal said the administration would spend $1 billion to encourage districts to switch to “student-based budgeting,” or letting funds flow to students rather than schools.

The approach is considered essential for school choice to thrive. Yet the mechanics of the Trump administration making it happen are far from obvious, as we reported in March:

There’s a hitch in the budget proposal: Federal law spells out exactly how Title I funds must be distributed, through funding formulas that sends money to schools with many poor students.

“I do not see a legal way to spend a billion dollars on an incentive for weighted student funding through Title I,” said Nora Gordon, an associate professor of public policy at Georgetown University. “I think that would have to be a new competitive program.”

There are good reasons for the Trump administration not to rush into creating a program in which states compete for new federal funds, though. … Creating a new program would open the administration to criticism of overreach — which the Obama administration faced when it used the Race to the Top competition to get states to adopt its priorities.

It’s unclear from the Post’s report how the Trump administration is handling Gordon’s concerns. But the Post reports that the administration wants to use a competitive grant program — which it’s calling Furthering Options for Children to Unlock Success, or FOCUS — to redistribute $1 billion in Title I funds for poor students. That means the administration decided that an Obama-style incentive program is worth the potential risks.

The administration’s budget request would have to be fulfilled by Congress, so whether any of the cuts or new programs come to pass is anyone’s guess. Things are not proceeding normally in Washington, D.C., right now.

By the numbers

After reshaping itself to combat declining interest, Teach For America reports a rise in applications

PHOTO: Kayleigh Skinner
Memphis corps members of Teach For America participate in a leadership summit in last August.

Teach for America says its application numbers jumped by a significant number this year, reversing a three-year trend of declining interest in the program.

The organization’s CEO said in a blog post this week that nearly 49,000 people applied for the 2017 program, which places college graduates in low-income schools across the country after summer training — up from just 37,000 applicants last year.

“After three years of declining recruitment, our application numbers spiked this year, and we’re in a good position to meet our goals for corps size, maintaining the same high bar for admission that we always have,” Elisa Villanueva Beard wrote. The post was reported by Politico on Wednesday.

The news comes after significant shake-ups at the organization. One of TFA’s leaders left in late 2015, and the organization slashed its national staff by 15 percent last year. As applications fell over the last several years, it downsized in places like New York City and Memphis, decentralized its operations, and shifted its focus to attracting a more diverse corps with deeper ties to the locations where the program places new teachers. 

This year’s application numbers are still down from 2013, when 57,000 people applied for a position. But Villanueva Beard said the changes were working, and that “slightly more than half of 2017 applicants identify as a person of color.”