A year after starting to rotate teachers without permanent positions into different empty slots weekly, the Department of Education has settled on a way to evaluate them.

But the plan, hiring administrators to observe and coach the teachers in multiple placements, could be stymied if the department cannot find enough available evaluators who are up to the task.

Last year, when the city launched the rotation system for members of the Absent Teacher Reserve, it left up in the air the question of who would be responsible for evaluating them. Previously, ATRs were typically assigned to one school for the entire year, so principals could rate them as they did any other teacher on staff.

For almost all of the roughly 830 teachers in the pool at the end of last year, district superintendents ended up issuing the annual ratings with input from potentially dozens of principals who supervised each teacher — in most cases, without conducting the formal observations that teachers are required to receive each year.

But in Brooklyn, which had about 250 ATRs last year, the city took a different approach. It interviewed and selected five administrators who had also lost their positions to budget cuts or school closures to visit the teachers in their classrooms and give them feedback about their performance.

The “field supervisors” each took on a caseload of between 20 and 30 ATRs, observing them several times throughout the year and conducting four training sessions for them as well, according to department officials. Ultimately, the administrators rated each teacher as either satisfactory or unsatisfactory at the end of the year.

Union officials said they were initially skeptical that the administrators would balance support with tough feedback, given the city’s repeated demands that it be allowed to fire ATRs, whom former chancellor Joel Klein characterized as “teachers who either don’t care to, or can’t, find a job.”

But the officials said they received few complaints from teachers participating in the pilot and found that the administrators selected for the task largely were conducting the observations in good faith. Some ATRs whose first observations netted them an unsatisfactory rating even had that label reversed after follow-up observations documented progress, the officials said.

Ultimately, 60 of the ATRs evaluated last year were rated unsatisfactory overall, according to city data — a rate three times the citywide rate but hardly suggestive of a broad effort to push ATRs out of the system, particularly because a higher portion of teachers in the pool had previous U-ratings. Teachers who receive two U-ratings can be fired, and one low rating can make it harder for teachers to land a permanent position.

Now the city is planning to expand the observation system piloted in Brooklyn to the rest of the ATR pool, which today numbers about 1,822. The number is likely to shrink by the end of October, when schools set their student registers and more teachers receive offers for permanent positions.

Amy Arundell, a teachers union representative who oversees personnel matters, said the union is in favor of the pilot’s expansion.

“We’re supportive of the DOE creating a structure whereby folks in the excess pool get the same kind of support that all teachers across the system are supposed to get,” she said in a statement.

But union officials said they have been told the expansion is contingent on having enough administrators on hand who are both capable of the unorthodox task and contractually able to complete it.

Currently, there are 200 administrators in excess in the city, according to department officials.

Teachers in the ATR pool who were rated last year said they were not satisfied with the way they were evaluated.

One teacher who attended a hiring fair on Thursday said she did not remember being observed at all, but she received a satisfactory rating anyway — a designation she said felt completely arbitrary.

“Whatever they ask you to do, you just do it,” she said of the assignment process, which had her teaching middle school at times and preparing her own lesson plans when someone she substituted for did not leave instructions.

A middle school teacher starting his third year in the pool said he was part of the Brooklyn pilot last year. Even with an observation, he said, he found the system “unfair” because his supervisor had little information about his performance before issuing a satisfactory rating.

The strong mark came “only because I happened to get an honors class on the day that he observed me,” said the teacher, who asked to remain anonymous. He said he was observed twice but not offered any extra training.

Connie Pankratz, a department spokeswoman, said the city was short on manpower last year, so it focused the observations on teachers who had been reported for problems such as poor attendance. This year, the department intends to observe every ATR at least once, she said. But she said she did not yet have concrete details on how the program would roll out and who would staff it.

The evaluation strategy, which Pankratz said had been developed with the union’s support addresses one problem posed by the rotation system: Each ATR does not have a regular set of supervisors. But it does not tackle some of the other system’s other challenges. Because many of them receive assignments outside of their license areas, or at schools that only need help with administrative tasks such as record-keeping, judging their teaching quality is trickier.

That reality was reflected in the ratings issued last year, union officials said, noting that most of the U-ratings were for attendance issues. A low rating attributed to incompetence in the classroom would be easily challenged if the city could not show the teacher had been observed and given chances to improve, or if the teacher was handling classes he or she was not licensed to teach, the officials said.

The teacher who participated in the pilot last year said the lack of formal observations and support might have been for the best, because he said during the year he encountered some capricious principals and was sometimes placed in jobs outside his license area.

“One principal didn’t believe in ATRs, so he had me sit with a teacher all day long,” the teacher said. “You could be an elementary teacher and still be put in a seventh-grade class.”