U.S. Education Secretary Arne Duncan said Tuesday that the federal government is obligated to intervene if states fail to address the rising number of students who are boycotting mandated annual exams.

Duncan’s comments come as an “opt out” advocacy group in New York reports that more than 184,000 students statewide out of about 1.1 million eligible test takers refused to take last week’s English exams. In New York City, nearly 3,100 students out of about 420,000 test takers opted out, according to the group. (Math testing begins Wednesday.)

Last year, 49,000 students statewide did not take the English exams, while just over 1,900 New York City students sat out the tests, state officials have said. State and city officials have not yet released their own opt-out counts this year or verified those of the advocacy group, United to Counter the Core, whose unofficial tally is based on information from district superintendents, school employees, and media reports.

Those estimates suggesting that more than 15 percent of students refused to take the tests have raised questions about the consequences for districts. Federal law requires all students in grades three to eight to take annual tests, and officials have said districts could face sanctions if fewer than 95 percent of students participate. On Tuesday, when asked whether states with many test boycotters would face consequences, Duncan said he expected states to make sure districts get enough students take the tests.

“We think most states will do that,” Duncan said during a discussion at the Education Writers Association conference in Chicago. “If states don’t do that, then we have an obligation to step in.”

Duncan also said that students in some states are tested too much, and acknowledged that the exams are challenging for many students. But he argued that annual standardized exams are essential for tracking student progress and monitoring the score gap between different student groups.

He also said the tests are “just not a traumatic event” for his children, who attend public school in Virginia.

“It’s just part of most kids’ education growing up,” he said. “Sometimes the adults make a big deal and that creates some trauma for the kids.”

A federal education department spokeswoman said last week that the agency could withhold funding from states if some of their districts have too few students take the exams, but that it has not yet done so because states have addressed the issue on their own.

State education department spokesman Jonathan Burman said in an email Tuesday that the “the feds are discussing the possibility of imposing penalties for failing to hit participation rate targets.” He added that the state is also expected to “consider imposing sanctions” on districts that fail to meet the 95 percent threshold, which could include withholding money “in the most egregious cases.” (Opt-out advocates have questioned the state’s interpretation of the testing law, and say that sanctions are only allowed after a district falls below the 95 percent level for three years.)

Meanwhile, Board of Regents Chancellor Merryl Tisch has said she thinks it would be wrong to “punish” districts because of their opt-out numbers.

“I would say to everyone who wants to punish the school districts: hold them to standards, set high expectations, hold them accountable, but punishing them?” she told the Buffalo News last week. “Really, are you kidding me?”

Sign up for Chalkbeat’s morning newsletter here for a daily dose of New York City education news.