Major districts of two minds on temporary teacher evaluation reprieve

Colorado school districts got a temporary reprieve this year in using student academic growth to evaluate teachers, and major districts are split into two camps about how to use that flexibility.

Just over half of Colorado’s 20 largest districts will continue to use student academic growth data for 50 percent of teacher evaluations.

Another half-dozen large districts are taking advantage of Senate Bill 14-165 and won’t be applying student growth when evaluating their teachers.

Among the 50-percent districts who will use growth data are the Douglas County and Jefferson County schools. Major districts choosing to skip growth for a year include Adams 12-Five Star, Aurora, Boulder, Cherry Creek and Denver.

The first group of districts employs about 19,000 teachers, while about 17,000 teachers work in the districts that won’t be using growth.

The information comes from a Chalkbeat Colorado survey of the state’s 20 largest districts, conducted to find out how each is using evaluation flexibility. Those districts employ about 72 percent of the state’s roughly 54,000 teachers.

District decisions appeared to be driven by a combination of three factors – a district’s level of confidence in its evaluation system, a desire to refine its systems further, and concern about changing its evaluation system for only one year.

“We decided we were just going ahead to push forward, instead of taking a step back,” said Marne Milyard, a principal in the Pueblo City Schools, which is sticking with 50 percent.

On the other side, Tracy Dorland of Adams 12 said, “We’re grateful for the gift of time, and we plan to use it well.” Dorland is the district’s chief academic officer.

Sen. Mike Johnston, author of the flexibility law, isn’t surprised that different districts have taken different directions. “I think that reflects what our intent was,” the Denver Democrat said.

“A lot of folks we talked to had done the advance work and were prepared” to use growth measures for evaluation this year. But, “It makes a lot of sense that a lot of people [in other districts] would say, ‘Let’s test the system first.’”

“We hope it will lead to a good year of reflection and work,” said Johnston, who’s also the author of the 2010 law that created the new evaluation system.

Long timeline, stops and starts for evaluation rollout

The teacher and principal evaluation systems required by Johnston’s Senate Bill 10-191 were implemented in all districts during the 2013-14 school year. Educator evaluations were based 50 percent on classroom and professional practice and 50 percent on what are called “measures of student learning” – academic growth as measured by a variety of tests. Ratings of partially effective or ineffective didn’t count against future loss of non-probationary status.

This school year and this year only, thanks to SB 14-165, districts can choose to continue the 50-50 system, use a smaller percentage for student growth or base evaluations solely on professional practice. Districts do have to calculate and record student growth measures for teachers even if they’re not used in evaluations. And, low ratings do count against possible loss of non-probationary status. (The flexibility was granted by the legislature partly because of the “data gap” that will be created by moving to the new CMAS testing system next spring.)

The evaluation system is supposed to return to its original design in the 2015-16 school year, with evaluations based 50-50 on practice and student growth and with low ratings counted against teachers.

What the 50-percenters say

Some of the districts that decided to stick with the 50-50 evaluation system did so for continuity.

“We know it’s going back to having that [growth] requirement a year from now,” said David Peak, assistant superintendent for human resources in the Academy district near Colorado Springs.

Peak said his district started working on the issue three years ago and has student growth measurements “that we believe are valid and reliable, but we recognize there’s still work to be done in that area.”

Ruth DeCrescentis, chief human resources officer for the Brighton schools, said, “We’re going to go ahead with 50 percent because we’re ready. … I think people are pretty comfortable.”

“We had all components for Senate Bill 191 in place last year,” said Todd Engels, executive director of educator effectiveness for the Jeffco schools. But, he added, “There are pieces we’re continuing to refine.”

The district decided, “We need to have some consistency for evaluating our educators” and didn’t want to change the rules after having used student growth in 2013-14, Engels said.

Some 50-percent districts nevertheless are adding pieces of local flexibility to evaluation practices.

Colorado Springs District 11 is using only what are called “collective measures” – data such as school performance framework ratings – not individual classroom growth measures – to evaluate teachers.

And while the Douglas County Schools will use 50 percent growth for evaluation, that data won’t be used in the district’s performance pay system this year. (Use of performance pay is a local decision and isn’t part of state evaluation law.)

Thoughts from the 0 percent districts

“We believe there is still a great deal of work to do and learning to be accomplished before we introduce student achievement measures to the evaluation construct,” said Damon Smith, Aurora chief personnel officer.

Judy Skupa, Cherry Creek’s assistant superintendent of performance improvement, sounded a similar note, saying, “There are many unknowns right now due to the transition of state assessments, and this provides us with more time to determine how new state assessments will inform teacher and principal effectiveness. Also, we are beginning to develop a deeper understanding of quality assessments at the classroom level; the ‘14-15 school year provides a time for us to use this deeper understanding to pilot different assessments across the district and to use that information to make the best decision possible.”

Denver Public Schools has been a leader in developing evaluation systems but, under an agreement with the Denver Classroom Teachers Association, opted for 0 percent this year. The district is doing more work to develop usable “individually attributable student growth data” – student performance information that can be connected to individual teachers, not just to the performance of whole schools or the district.

“Because we don’t have other measures of individually attributable student growth for enough of our teachers, we decided to take advantage of the flexibility afforded by SB 165,” spokeswoman Nancy Mitchell said.

Theresa Myers, a spokeswoman for the Greeley schools, seemed to sum up the feelings of many districts by saying, “The reason was to give teachers one more year to learn about the system, to understand what supporting materials they will need to provide and to have one more year to feel comfortable with all the new evaluation entails.”

The outliers

The flexibility law allows districts to use any percentage between 0 and 50 for student growth this year. The Mesa 51 district in Grand Junction is the only top-20 district that isn’t doing all or nothing, and it is using 25 percent. “It’s the same rationale as those who are dropping it to zero percent; we want another year of experience with the process with the belief that we’re still honing our skills,” said spokesman Dan Dougherty.

Katy Anthes, director of educator effectiveness for the Colorado Department of Education, said she’s heard that several districts have chosen either 0 or 50 percent but doesn’t have a count because the flexibility law doesn’t require districts to tell CDE which route they’ve chosen.

Linda Barker, director of teaching and learning for the Colorado Education Association, said she’s heard that at least one smaller district, Meeker, is using 25 percent.

The flexibility law doesn’t set a deadline by which districts had to decide. Among top-20 districts, the Thompson schools still haven’t made a decision.

How student growth is measured

It’s a common misconception that Colorado’s evaluation system rates teachers based on student scores on statewide tests. That’s only half the story – remember that half of evaluations are based on classroom and professional practice.

And districts can measure the student growth half with a variety of data, including growth as measured by state tests, teacher-designed classroom tests, assessments provided by outside vendors and even growth in the ratings provided by the school and district performance frameworks that are used for accreditation and rating. Many districts are working on creation of student learning objectives that can be used to measure academic growth. From those different measures districts can choose their own mix for their evaluation systems.

The Colorado Education Initiative recently issued a report on districts practices, based on a study of what 53 districts did during the 2013-14 school year.

“In most cases, districts rely on teachers to develop their own assessments to measure student growth rather than relying solely on state or vendor assessments that test only a relatively narrow range of content,” the report concluded, although it did find a somewhat heavier reliance on collective growth measures by larger districts.

The graphic below summarizes the report’s findings. Get links to a summary and the full report here. (MSL stands for “measures of student learning.”)

RANDA to the rescue?

The Education Initiative’s report noted that “The burden of data collection and scoring has been substantial, and many districts would welcome more streamlined and cost-effective data tracking systems.”

Officials at CDE are trying to provide some help with that in the form of a $2 million online data system that allows principals and administrators to enter and track evaluation data. For now the system is available only to districts signed up for the state’s model evaluation system, used by about 90 percent of districts (although not some of the larger ones).

The department says more than 80 districts already are using what’s nicknamed RANDA (after the name of the vendor) but which is formally called the Colorado State Model Performance Management System. Learn more about RANDA here.

Evaluation implementation a big challenge

The feelings of many administrators and teachers can be summed up by this sentence from the initiative’s report: “Developing, implementing, reviewing, and refining an MSL system takes an enormous amount of time and energy, both for administrators and educators.”

The CEA’s Barker said districts are “appreciating that they have this year of flexibility” but cautioned that “one year isn’t going to give us enough data.”

“This is a huge learning curve for all of us. … It’s going to take time to fully implement this.”