There's a well-known education research textbook by three distinguished scholars at Harvard entitled By Design. Judy Singer, one of the authors, once told me that the working title for the book, rejected by Harvard University Press, was Bungled by Design. That title conveyed the key message of the book, which is that, when it comes to education research, you can't fix by analysis what you bungled by design. The design of a research study dictates what a researcher can plausibly ask, and the credibility of the claims about what is being studied. The recently-released NYU study of the New York City Principal Leadership Academy comparing graduates of the Aspiring Principals Program to other new NYC principals is, in my view, bungled by design. This is not a knock on the authors, each of whom I know and respect a great deal. Rather, it reflects the fact that the NYU researchers were brought in to study the Aspiring Principals Program of the Leadership Academy long after critical design decisions about how to evaluate the impact of the program were made—either by omission or commission. The three key limitations I raise here pertain to selection mechanisms that ideally would have been observed by the researchers. The inability to understand and model these selection processes undermines the objective of isolating the effect of the Aspiring Principals Program on student outcomes. (See the comments of Sean Corcoran, lead author of the report, on selection issues here.)
The outlook for city teachers without positions hasn't brightened much in the last month, even with the external hiring freeze meant to help them land jobs. Just about 300 of the teachers in the Absent Teacher Reserve found new jobs in the last month. That leaves about 2,000 ATRs on the Department of Education's payroll with just weeks before school starts. And their chances of finding a spot might be tough: Though the system has 1,800 openings, some principals are signaling they are hoping to fill their spots with outside teachers, rather than hire jobless teachers from within the department. At a hiring fair in Queens on Tuesday night, principals snapped up eligible teachers in minutes, reported a teacher named Jenn in a comment at GothamSchools. But other principals appear to be hanging onto the hope that they'll soon be able to have their pick of aspiring teachers, despite Schools Chancellor Joel Klein's warning that the freeze would not be lifted soon.
More black and Hispanic students are taking the SAT, but is that just because of overall demographic shifts? A reader asks for overall enrollment trends by race. The data show that the numbers of black and Hispanic students in the city is not rising. The black population has been declining while the Hispanic population is also declining, though less rapidly. The number of Asian students in city schools is rising. This is according to both city figures on public school enrollment and Census estimates on the size of the school-aged population.* I spent several months last year exploring the public school enrollment data, which contains all kinds of mysteries (one: white enrollment in public schools has declined while the white school-aged population, by Census estimates, which are imperfect, is rising). Alas I only completed my digging just as the New York Sun was closing, and it's never seen the light of day — until now! Here's a chart I put together last year, using city data, followed by a chart using the Census's school-aged population estimates:
Yesterday, the College Board released its annual report on the SAT, and New York City was quick to follow suit with data on the performance of NYC high school students on the SAT. Citywide average scores fell a few points, at the same time that the numbers of Black and Hispanic students taking the SAT increased. Writing in the Daily News, Rachel Monahan summarized the DOE spin, courtesy of DOE spokesman Andy Jacob: (a) More Black and Hispanic students took the SAT, and fewer white students did; (b) the increasing numbers of SAT-takers are less likely to be high performers than SAT-takers in the past; (c) therefore, let's focus on the increased representativeness of the test-taking group, and ignore the fact that scores fell among Blacks and Hispanics, and that the achievement gap is still huge. I don't think that we should pay too much attention to single-year changes in test scores of any kind, and especially the SAT, which commenter CarolineSF points out are taken by a self-selected group of high school students. But this year's snapshot nevertheless reveals some hard truths about the performance of New York City's high school students. Let's address the representativeness issue first. Is there evidence that the rising numbers of Black and Hispanic students taking the SAT reflects a dramatic change in the kinds of students who are taking the SAT? Can we explain the falling average Black and Hispanic SAT scores as reflecting a new group of low-performing NYC high school students striving to get into college?
Data courtesy of the city Department of Education. As promised, here's some more detail on who takes the SAT in the city — broken down by race and painted as a picture over time. The number of black students taking the SAT is now at 10,438, up from 6,763 in 2002. The increase among Hispanic students is even more pronounced: From 5,400 in 2002 to 11,414 in 2009. Scores for both groups in 2009 were stuck in the low 400's on each subject matter. That would make about an 825 out of 1600 on the old scale, which included just math and reading and no writing. Also, curiously, the number of white students taking the SAT dropped in the city this year (though it's still above the 2002 number) as in America. On the 1600 math-and-reading scale, white students this year scored 1,031 on average. And everyone seems to score about the same on the new writing test as on the math and reading test. But, like we keep (unconvincingly?) saying, we're on blog-vacation! So please help us out by pointing out the trends you see in the comments. (And check out Caroline's point about not putting too much stock in changes in the overall averages.)
New York City students followed statewide patterns on the SAT and AP college prep exams, according to College Board data released today. We had a more diverse pool of test-takers than last year — more black and Hispanic students took both the SAT and AP exams, and more passed the AP; lower SAT scores on average since last year; and a higher (though still dismally low) number of students passing the AP exams. The city Department of Education summed up the results in a PDF (embedded below the jump). I asked for a more specific breakdown — average scores from past years and average scores broken down by race — and will provide that when it arrives. You can read the national results here and the New York results here. (Would that we already had the forthcoming IBO education team to give us the straight story from the get-go!)