New York City has been trying to help struggling schools by partnering them with nonprofits that provide counseling and health services. A Detroit school recently added a washing machine to make sure students have clean clothes. A Tennessee superintendent just petitioned the state for more funding to offer similar help to students and families.

The strategy, often referred to as the “community schools” model or “wraparound services,” has been embraced by districts across the country. It also makes intuitive sense to help kids in class by directly dealing with out-of-school factors, like poverty, that affect learning.

So do school-based efforts to counter the harmful effects of poverty lead to measurable academic gains?

Here’s what we know: Research shows that these efforts often do help learning, but in a number of cases they don’t seem to have any effect — and it’s not clear why efforts sometimes succeed and sometimes don’t.

The impact on academics is promising

Child Trends, a research group, recently compiled and analyzed the results of 19 rigorous studies that tried to isolate the effects of efforts to improve students’ mental and physical health, offer counseling services, add after-school programs, provide direct social services to families in need, and other similar programs.

Examples include the national Communities in Schools and Boston’s City Connects programs, which place site coordinators in schools to connect students and families to those resources.

When looking at the effect of wraparound services on grades and test scores, those 19 studies come to a mix of positive and inconclusive findings. Results were a bit more positive in math than in English, which is common in education research.

There was also variation within programs, like Communities in Schools, which has become the most evaluated wraparound-style initiative. Separate studies have shown that the program produced test score gains in Chicago and Wichita, but not Austin or Jacksonville. A recent national evaluation focusing on Texas and North Carolina found a mix of outcomes.

One notable finding: across the 19 studies, there are virtually no cases where students appear to do worse thanks to the programs, the review notes. The researchers conclude that the approach is “promising but not yet proven.”

Not included in the review were a few initial evaluations of New York City’s community schools-based turnaround program, which included extending the school day. One analysis found that the program actually seemed to reduce high school graduation rates relative to similar schools that did not participate, and had no effects on elementary or middle school test scores. But another study using a different approach found that the initiative did lead to moderate test score gains.

The impact on attendance, behavior, and other outcomes is inconsistent

One surprising aspect of the research on these wraparound services: there aren’t consistent findings about how the programs affect things other than academics.

In a handful of studies in the Child Trends that examined other outcomes, most found no effects on students’ attendance, behavior, engagement in school, or social-emotional outcomes. Still, a few studies found positive effects and, again, negative ones were quite rare.

One recent paper, not included in the Child Trends review, found that a wraparound initiative in Massachusetts led to substantial gains in students’ math and English test scores. That program made no apparent impact on students’ attendance, their likelihood of being held back a grade, or suspension rates, though.

What makes a program work?

Frustratingly for policymakers, it’s not clear.

The Child Trends report suggests providing community schools with substantial resources over several years is most likely to lead to success. But it concludes that there’s a “lack of evidence regarding the concrete elements that make different models successful or how they must be implemented.”

Meanwhile, there appears to be stronger evidence for the academic benefits of direct anti-poverty programs that are separate from schools. The earned income tax credit, health insurance, child tax credit, food stamps, and simply giving cash to low-income families have all been linked to better outcomes in schools for children.

Finally, many would argue these sorts of wraparound services and anti-poverty programs are worthwhile regardless of students’ short-term academic gains.

Elaine Weiss, who led a group that supported wraparound services, previously told Chalkbeat that the approaches have intrinsic value.

“Don’t we all agree that having kids who have access to mental and physical health care, regular nutritious meals, and quality, safe after-school and summer programs is inherently a good thing?” she asked.