A group of teachers boycotted a required part of their school’s teacher evaluation plan on Thursday, breaking from the city education department, the teachers union, and their principal and raising questions about whether they will face disciplinary action.

Standing in a park across the street from International High School at Prospect Heights, more than a dozen teachers held up signs and criticized the increased role that tests have played in their school. They argued that a city-created reading and writing test now being administered in city high schools was developed with little consideration for their students—most of whom recently moved to the United States and are still learning English.

“We believe that the tests should be designed for the students who are being tested,” said Steve Watson, a math teacher and the school’s union chapter leader. “And the students that are being tested are not standardized, so we do not believe that one size fits all.”

International High School at Prospect Heights is part of a network of schools serving immigrant students. Most students in the school had already opted out of the test, but the protesting teachers said they would refuse to give tests to those who still planned to take them today.

Still, their protest didn’t actually stop any students from taking the tests, according to Department of Education officials who said that the school’s principal, Nedda de Castro, still planned to administer the exams. (De Castro did not respond to requests for comment on Thursday morning.)

But the protest did reflect a new side of the anti-testing sentiment that has bubbled up across the city this year.

Unlike other anti-testing demonstrations, the International High School protest was focused not on state tests but on a new, separate test required to measure student learning for use in the new teacher evaluation formulas.

Similar tests were given earlier in the year to establish “baselines” for student growth, prompting pushback in some schools. Elementary school teachers refused to administer some tests to their youngest students, and some students at Stuyvesant High School publicly refused to take them.

Those tests were less significant than the ones teachers were protesting today because they could be replaced with other kinds of student progress data, like student work from the beginning of the year. The exams the protesting teachers would have administered today, on the other hand, are schools’ only option for assessing student growth for evaluations.

That put the boycott on shaky legal ground, since the framework for the teacher evaluation system is a part of the teachers union contract.

The United Federation of Teachers distanced itself from the event, though it was organized with help from the Movement of Rank and File Educators, a political caucus within the union. A union spokesman said that while the UFT agreed with the demonstrators’ anti-testing sentiments, teachers are “under a legal obligation” to abide by rules of the evaluation system.

“The UFT believes that our schools have been the victims of a testing culture that has focused far too much attention on test prep and too little on strategies that will actually lead to student learning,” the spokesman said. “However, this protest is not a union-sponsored event.”

A spokeswoman from the Department of Education said the city was looking into the situation and potential disciplinary action against the protesting teachers.

Ironically, though the teachers were defying the union and the city with their protest, they all have all raised similar concerns about over-testing students.

The rules of the teacher evaluation system were handed down last year by State Education Commissioner John King. Both the city and the teachers union have said they don’t like his decision to require the contested performance assessments, and the union and the city have both recently asked the state to allow some flexibility around how schools can measure teacher performance. But the state has placed the blame on the city, saying it could have chosen a different way to measure student learning, but opted for the performance assessments instead.

The protesting teachers have also sent a letter to schools Chancellor Carmen Fariña, asking that she allow schools to use different types of assessments that better fit their students’ language needs. In particular, they said, student learning should be based on portfolio-style assessments, which are intensive projects that require students to complete a series of assignments.

In her first months on the job, Fariña has spoken out about a need to better serve the city’s English Language Learners. But a department spokeswoman said that was unrelated to today’s protest.

“At the International High School at Prospect Heights, these assessments have no stakes for students, and as DOE policy mandates, these students should receive all the accommodations they require,” the spokeswoman said.