To our readers

Hey, we heard you. You had a lot of questions about TNReady. We found answers.

The news that Tennessee’s testing company scored some high school tests incorrectly this year uncorked a flood of questions about the validity of the state’s new standardized assessment.


Here are five things to know about the latest brouhaha over TNReady


We wanted to know how the brouhaha was impacting classrooms, so we asked our readers on Facebook.

You responded in droves.

We took your top concerns directly to the state Department of Education and asked for answers. Here’s what you wanted to know — and what we have learned:

Several readers asked why they should trust TNReady results, given the series of setbacks in the test’s first two years.

  • “I do not trust the results. We have had so many problems in the last few years, that I am suspicious of any results we do get. It bothers me greatly that the state uses these numbers to hold students and teachers and districts accountable, but they seem to be unable to deliver scores they believe are accurate in a timely manner.” —Rebecca Dickenson
  • “I no longer trust the accountability of the state nor its methods. My concern is if there is a teacher who has only one year of test data, how is it the same teacher shown multi-year growth when he or she had only last year of testing? This poses a huge concern.” —Mildred Williams  

Tennessee Department of Education: “TNReady is fully aligned to Tennessee’s academic standards, and every question has been reviewed, edited, and approved by Tennessee teachers through a rigorous review process. We also have quantitative checks and processes after a test is over to ensure student responses are reliable. While more than 99.9% of TNReady tests were scored accurately this year, we want to improve on that next year, and our vendor (Questar) is taking new quality assurance steps to make sure their programming is error-free. Also, this year, as soon as the scoring error on some of the English I, II and Integrated Math II EOCs was identified, scores were updated and all TNReady tests were re-reviewed and verified for full accuracy.”

Some teachers told us that, given the delay in score deliveries this spring, many students don’t think the results will arrive in time to affect their final grades next spring. Those teachers are struggling to get their students to buy in.

  • “After two years of TNReady, it still hasn’t counted for my students. Going into year three, I will once again tell them with a hopeful, straight face that it will count as part of their report card grades and implore them to try their best. I quietly wonder what reason they have to believe me, given recent history.” —Mike Stein
  • “I struggle to get students to buy in to the importance of trying their best on state tests because the students are confident that the scores won’t come back in time to affect their grades (which has been the situation for several years now). The students see zero incentive for doing well.” —Nicole Mayfield

TDOE: “We believe that if districts and schools set the tone that performing your best on TNReady is important, then students will take the test seriously, regardless of whether TNReady factors into their grade. We should be able to expect our students will try and do their best at any academic exercise, whether or not it is graded. This is a value that is established through local communication from educators and leaders, and it will always be key to our test administration. We believe that when we share these messages and values celebrating the variety of accomplishments our students have made, taking advantage of TNReady’s scheduling flexibility to minimize disruption, focusing on strong standards-based instruction every day, sending positive messages around the importance of the variety of tests that students take, and sharing that students should always do their best then students will buy-in and TNReady will be successful.”

Other teachers asked what happens to writing scores for tests in English language arts.

  • “I can tell you that two years ago — when we first piloted the new writing test online — districts received not only every student’s scores (broken down by each of the four indicators) but also the actual student responses to each prompt. In my former district our supervisor shared them, and we analyzed them as a department. If you check with your principal, VP, or supervisors, there are some published “anchor papers” with scores available on edtools from this past year. It’s not a lot, but it’s more than we’ve had in the past. My hope is that if online continues, we’ll keep seeing the student responses in the future.” —Wj Gillespie II

TDOE: “The question appears to be referencing the process we had through the 2014-15 school year, when our writing assessment was separate. Since 2015-16, students’ writing responses on TNReady have been incorporated as part of their overall ELA score. Responses are scored based on our writing rubrics, and for educators, we have provided access to the “anchor papers” from the 2016-17 year, so they can see how students’ responses were scored based on the writing rubric, which can help them inform the feedback they give their students.”

On that same issue of writing scores, one teacher referenced the hiring of scorers off of Craigslist. We asked the state if that’s true.

  • “I continue to be curious about our ELA writing scores. Each year we are required to use state writing rubrics, attend PD related to the state’s four types of writing, etc etc…and yet our scores never come back. Students spend hours taking the writing portion of the test, scorers are hired off Craig’s list…, and yet we never actually get the scores back. It seems like every year this is swept under the rug. Where do these writing tests go?” —Elizabeth Faison Clifton

TDOE: “Questar does not use Craigslist. Several years ago, another assessment company supposedly posted advertisements on Craigslist, but Questar does not. We provide opportunities for our educators to be involved in developing our test, and we also encourage Tennessee teachers to apply to hand-score TNReady. To be eligible, each applicant must provide proof of a four-year college degree, and preference is given to classroom teachers. As part of the interview process, an applicant would have to hand-score several items for review and evaluation. Once hired, each scorer is trained based on materials that Tennessee teachers and the department approve — and which are assembled from responses given by Tennessee students on the exam — and scorers are regularly refreshed and “recalibrated” on scoring guidelines. Each writing response is scored at least twice; if those two responses differ significantly, they are sent to a third scorer. Each day, the department reads behind a sample of essays to ensure hand-scorers are adhering to the criteria set by our teachers. Any scores that do not align are thrown out, and those scorers are retrained. Any scorer who does not meet our and Questar’s standards is released from scoring TNReady.”

Finally, readers expressed a lot of concern about the complexity behind growth scores known as TVAAS, which are based on TNReady results and which go into teachers’ evaluations. We asked the state for a simple explanation.

  • “What formula is used in calculating the overall score for TVAAS when fallacies were determined as a result? My performance is weighed heavily on the state TVAAS score which is why this type of error has occurred before. This is quite disturbing. Teachers work tirelessly to ensure student achievement is a success; however, testing to measure performance seems to not be working.” —Mildred Williams  
  • “No one can give me the formula for how my students’ scores are calculated to create my score in TVAAS. How is (t)hat transparency? Yet, I’m required, constantly, to “prove” myself with documentation of education, observations, professional development and the like; all in originals, of course, to numerous overseeing bodies.” —Rachel Bernstein Kannady
  • “I find it ludicrous that data from these tests are used to evaluate MY performance when I get little to no control over most of the variables regarding the test. How could a miscalculated, misinformed, and (for all I know) incomprehensible test demonstrate what my students have learned!? And don’t even get me started on that fact that the rigor of the tests was increased ten-fold, yet not scaffolded in.” —Nicole Mayfield

TDOE: “TVAAS is statistically valid and reliable, and we follow the recommendations outlined by the American Educational Research Association (AERA) on value-added measures. Conceptually, TVAAS looks at how students have performed historically on TCAP and TNReady and compares their performance to their peers who have had similar past performance. If students tended to grow at about the same rate as their peers across the state — the expected amount of growth — they would earn a 3. If students tended to grow faster than their peers, they would earn a 4 or a 5, depending on the amount of progress they showed. If they tended to not show as much growth as their peers, they would earn a 1 or a 2. The model itself is sophisticated and complex to be as fair and nuanced as possible for each teacher’s situation, and we are working with our educator preparation providers as well as district leaders to provide more training on specifically how the model calculates scores. Tennessee educators also have access to a TVAAS user support team that can answer any specific questions about their TVAAS data, including how the data was analyzed.

Because TVAAS always looks at relative growth from year to year, not absolute test scores, it can be stable through transitions — and that is what we saw this year. Students can still grow, even if their overall proficiency level is now different. You can think about it like a running race. If you used to finish a 5K at about the same time as 10 other students, and all 10 students made the same shift to a new race at the same time with the same amount of time to prepare, you should finish the new race at about the same time. If you finished ahead of the group’s average time, you grew faster than your peers. If you lagged behind everyone, that would indicate you did not grow as much as was expected.  Because students’ performance will be compared to the performance of their peers and because their peers are making the transition at the same time, drops in statewide proficiency rates resulting from increased rigor of the new assessments had no impact on the ability of teachers, schools, and districts to earn strong TVAAS scores. Transitions to higher standards and expectations do not change the fact that we still want all students in a district to make a full year’s worth of growth, relative to their peers who are all experiencing the same transition.”

Reporter Laura Faith Kebede contributed to this report.

Charter strike

Chicago charter files federal labor complaint against union over strike

PHOTO: Yana Kunichoff / Chalkbeat Chicago
Chicago Alderman Ed Burke, left, met Dec. 7, 2018, with striking Acero teachers and their supporters, who were protesting at his office.

As the acrimonious teacher strike against Acero charter schools wound down its fourth day, both sides ratcheted up pressure, neither giving any indication of backing down.

The charter network sought a court order to halt the strike, and filed a federal complaint claiming that the strike was illegal.

Meanwhile, powerful Alderman Ed Burke, who represents areas heavy with Acero schools, addressed strikers who had marched into his office Friday.

“My heart is with you,” Burke told them. He promised to speak with Acero CEO Richard Rodriguez in an effort to end the strike before Monday, according to both Burke’s office and Acero.

Some 30 teachers and parents wedged into the foyer of Burke’s office between a lit-up Christmas tree and a statute of a horse wearing a green beanie labeled “Ald. Ed Burke.”

They demanded that he use his clout to pressure Rodriguez to agree to teachers’ contract demands, among them smaller class sizes and better compensation for teachers and paraprofessionals. Later Friday, Acero issued a statement confirming that the two, political allies, had met. The network did not explain the content or nature of the discussion.

About 500 teachers have been striking since Tuesday, with 7,500 students out of school. Seven of Acero’s 15 schools are in Burke’s ward.

Acero filed an unfair labor practices complaint against the Chicago Teachers Union and is appealing to the National Labor Relations Board to halt the strike. The charter management organization also sought a temporary restraining order to force teachers back to work. You can read the NLRB complaint below.

In response, CTU President Jesse Sharkey said in a press release, “Acero’s management is desperate and our pressure is working.” He insisted that the strike is a legal protest over wages and working conditions.

In response to strikers’ accusations that Rodriguez is uninvolved in the negotiations, Acero also issued a statement insisting that Rodriguez had met with management negotiators throughout the talks. Union officials have complained of Rodriguez being absent from the bargaining table.

Acero’s roots

Acero, once the nation’s largest Hispanic charter school operator, sprang from a community organizing tool to build Latino political power on Chicago’s Southwest side.

The history of Acero illustrates how charter schools in Chicago are intertwined in local politics, and how their growth would have been impossible without political support.

The United Neighborhood Organization was founded in 1984 by a Jesuit priest who recognized the struggle of immigrants in Chicago’s fast-growing Mexican-American community. Soon a South Side community organizer named Danny Solis joined and turned the organization’s focus first to local school politics and eventually to citywide influence.

Over the years, UNO’s power in neighborhoods grew as it nurtured local leaders like Juan Rangel, who eventually became CEO of the network. Both Rangel and Solis also ran for aldermanic positions, with Solis eventually winning an appointment in 1995 as alderman of the 25th ward, which encompassed the Pilsen neighborhood.

Rangel, meanwhile, had worked his way to the head of UNO just as then-Mayor Richard Daley and his school leadership team were ushering in an era of school choice in Chicago, and looking for community groups to take up the mantle.

“When charters emerged, UNO was one of the first entries into the charter market,” said Stephanie Farmer, a professor of sociology at Roosevelt University who researches charter school finance. “They did work their political connections to get state funding.”

UNO first proposed two charter schools in 1997.  Two decades later, it runs 15 schools spread across both the Southwest and Northwest sides of the city.

Enter Ed Burke. Halfway through an ambitious construction project for a new campus, UNO ran out of money and was forced to turn to its political allies, among them Burke, who helped the network get a $65 million low-interest loan from bankers. Several years later, Rangel supported Burke’s brother in his run for an Illinois House seat.

Farmer called this a clear example of the benefits of political patronage, without which Acero could not have grown as much as it has.

“They became patronage benefactors. It was both a way for UNO to build political power and then also a way for Burke to solidify his relations with the Latino political machine,” she said. “They were the only [charter school] who got as much state money as they did for the buildings.”

Rangel’s tenure at UNO ended abruptly and in disgrace. Accused of nepotism and misusing public funds, and under investigation by the Securities and Exchange Commission, he quit. In 2017, UNO rebranded itself as Acero in an effort to distance itself from its former leader’s misdeeds.

Today, charters in Chicago face a harsher climate than they did during Acero’s initial expansion.

Chicago Public Schools recommended this week that the school board deny all new charter applications for the next school year, bending to the political tide rising against the independently operated public schools. And the state’s new governor, Democratic businessman J.B. Pritzker, said while campaigning that he supported a moratorium on new charters.

But Burke’s ability to call Acero’s CEO and encourage him to come to an agreement shows that politics may still play a significant role in the charter industry.

It also shows a more critical turn both toward machine politics and education in Chicago, Farmer said,  “The strikers are highlighting that Burke’s machine doesn’t work for the ward’s children.”



money matters

Indiana Gov. Eric Holcomb proposes pumping brakes on teacher pay, cutting performance bonuses

PHOTO: Shaina Cavazos/Chalkbeat

As educators and lawmakers call for increases in teacher pay, Indiana Gov. Eric Holcomb is suggesting a far more measured approach, potentially tempering expectations that meaningful raises could be possible in the coming two years.

In his legislative agenda announced Thursday, Holcomb proposed devoting 2019 to studying teacher pay — how much money would be needed, how the state can ensure districts direct funds to salaries, and what amount makes Indiana competitive with surrounding states. He suggested that actual raises might not come until following years.

“We want to do it right, and it’s a big number. We have some ground to make up here,” Holcomb, a Republican entering his third year as governor, said. “And we want to … make sure we are showing local communities that the state is investing in them. We want them to have local control.”

It’s not clear if lawmakers will go along with this plan, however, as some have already made strong calls for upping teacher pay, which even by conservative estimates would probably cost the state hundreds of millions of dollars.

Democrat Phil GiaQuinta, the House Minority Leader from Fort Wayne, said the governor should take bolder steps to “increase teacher pay right now. We don’t need to study this issue before making a final decision.”

With the upcoming year projected to have a tight budget and little new revenue available, Holcomb also proposed eliminating $30 million in annual performance bonuses for teachers, and he is not supporting an increase in state money for pre-Kindergarten.

Instead, Holcomb is suggesting to split the $30 million bonus pool in two other efforts that his administration said are more equitable and impactful: sending about $20 million per year in overall state aid for districts, and putting the remaining $10 million per year toward increasing teachers’ school supply tax credit, from $100 to $500.

His decision to not support an increase in pre-K funding is a departure from years past, and flies in the face of advocates calling for Indiana to make pre-K accessible statewide to 4-year-olds from low-income families. In 2017, Holcomb was a key force in the decision to nearly double the state’s investment in its pre-K voucher program, known as On My Way Pre-K.

Essentially, officials with the Holcomb administration said Indiana can do more with the $20 million per year that it’s already spending on pre-K. One goal would be adding 500 students to the program each year by widening application windows and creating a website where families can apply for the vouchers, rather than in person during business hours. The program, currently serving about 3,000 students across 20 counties, is heading into the fourth year of its five-year pilot.

Officials from Holcomb’s office said that available federal dollars have increased from prior years, putting less pressure on the state to set aside more money. The state is also pursuing another federal grant, which could bring in up to $10 million to help preschool providers identify and address gaps in their current offerings.

Then, in 2021, officials would ask for more state dollars to expand the program to more counties and families, Holcomb’s staff said. Last month, House Speaker Brian Bosma said he would support adding more money to the program so long as it was focused on low-income families, but he also raised the possibility that the state might have to rely on federal funding.

Lawmakers can choose whether to follow Holcomb’s lead, but the governor doesn’t craft legislation. He can, however, veto bills at the end of the General Assembly’s session.

Holcomb, a Republican entering his third year as governor, did not outline exactly what he wants to spend on K-12 through 2021, deferring those specifics until the state releases its final revenue forecast of the year, although he did call for adding dollars to the state’s overall school funding. A budget proposal from the governor’s office is expected in early January.

The governor also wants to:

  • Set aside about $3 million per year to fund computer science training for teachers.
  • Move up the date to appoint the state’s next schools chief, from 2025 to 2021.
  • Require students to enroll in a one-semester class as freshman or sophomores that introduces them to college and career planning.
  • Change the career and technical education funding formula to favor classes that result in students earning a certificate or credential before they graduate.
  • Loosen the rules for workplace specialist licenses, which allow people with experience in skilled trades to become high school CTE teachers.

The rest of Holcomb’s agenda revolved around workforce development, expanding the state’s trail system, combating the health and mental effects of substance abuse, and creating a hate crimes law.

Read more from Chalkbeat: ‘Indiana’s war on teachers is winning’: Here’s what superintendents say is causing teacher shortages

Indiana’s push to raise teacher pay is creating some unlikely allies

Indiana faces a tight budget in 2019, lawmakers say. Will expanding pre-K be in the cards?