New York City’s announcement this week that it is doubling down on community schools indicates a firm belief that the program is working. But is this model effective, and by what measures? New research from an initiative in Texas and North Carolina aims to help answer those questions — and finds mixed evidence.

The precise details of community schools vary from place to place, but they generally emphasize a holistic model, by addressing factors — poverty, health, behavior — that might impede academic success.

Previous studies of similar programs have pointed to positive impacts, but the latest analyses, conducted by the research group MDRC and released last month, offered more tepid results.

The research focused on an initiative known as Communities in Schools, or CIS, which places “site coordinators” in schools to help connect students and schools with a variety of services, including mental and physical health services, family engagement, academic interventions, and college prep. The program operates in over 2,000 schools across 25 states. (CIS is not involved in the New York City program.)

The first MDRC report examined schools that participated in the program’s whole schools model that provides low-intensity services to all students in a given school; examples include “making clothing or school supplies available to students, organizing a school-wide career fair, or hosting a financial aid workshop for twelfth-graders.”

This intervention appeared to be effective in improving student attendance in elementary school and graduation in high school, but did not raise test scores — and in fact might have had a negative impact on middle schoolers’ math scores.

Another report, using a high-quality random assignment method, looked at CIS’s more intensive individualized supports, known as case management. “The site coordinator provides services appropriate to the student’s needs or connects the student to those services,” according to the paper.

The impacts were uneven: after two years, students receiving case management reported improved relationships with adults and peers and more positive attitudes about school, but had no better attendance or course grades. And the program actually caused a small increase in suspension rates.

Heather Clawson, the executive vice president of research at CIS, said that she viewed the results as “mixed,” but important.

“We’re very much a learning organization,” she told Chalkbeat. “All results are good results as long as they tell us something about what we’re doing well and what we need to improve.”

Clawson said that CIS has already made changes to its program based on MDRC studies that came out in 2015. (The latest analysis looks at schools prior to these relatively recent modifications.)

Meanwhile, as New York City plans to expand its program of community schools, the existing system has not been subject to rigorous external study, and early indicators are mixed. The city is partnering with the independent research group RAND to study its program; a look at the effect on students isn’t expected to be released for over a year.

A spokesperson for the New York City Department of Education pointed to previous research showing students can benefit from community schools in terms of achievement, high school graduation, and other outcomes. Indeed, past evidence — including national and Chicago-specific studies of CIS, an analysis of a Boston-based program, and a 2014 review of research — paints a largely encouraging picture.

Clawson said the goal of the program isn’t to raise test scores, but does expect it to improve other metrics: “We do believe behavior [and] attendance are outcomes we should be accountable for; test scores are a little more of a stretch.”

Elaine Weiss, of the Broader, Bolder coalition, which advocates for community schools, said the model is likely to improve student outcomes in the long run, but is important even if it does not.

“The services themselves are, of course worthwhile — don’t we all agree that having kids who have access to mental and physical health care, regular nutritious meals, and quality, safe afterschool and summer programs is inherently a good thing?”