It was supposed to be “a simple way to send poor kids to top colleges.”

Sending personalized college-application information and application fee waivers to high-achieving, low-income students pushed those students to attend more selective colleges, a 2013 study found.

Perhaps because it offered a cheap — just $6 per student! — way to solve a vexing social inequity, the study attracted a great deal of attention. The College Board, purveyor of the SAT, even decided to bring the idea to scale, launching their own effort to send encouragement and fee waivers to students nationwide.

“We can’t stand by as students, particularly low-income students, go off track and don’t pursue the opportunities they have earned,” College Board president David Coleman said in 2013.

Now, the results of that effort are in, and they don’t look anything like what many had hoped.

“Our interventions led to no change in the likelihood or sector of college enrollment,” the new paper, largely written by in-house researchers at the College Board, says.

The findings are an example of how even promising, research-backed ideas can wilt when they are expanded widely. And it’s a setback for the College Board, which under Coleman’s leadership has introduced a suite of high-profile initiatives meant to make the college process more equitable.

It’s also one piece of evidence that changing the college trajectories of America’s low-income students will require efforts more extensive than low-cost “nudges.”

“It’s a good lesson in thinking about the limitations of these kinds of interventions,” said Lindsay Page, a University of Pittsburgh researcher who has studied text messaging reminders to students. “Be cautious of the long-run benefits from $6 solutions.”

College Board outreach had virtually no impact on where students go to college

In 2016 and 2017, the College Board set out to encourage low- and middle-income students to apply to schools they otherwise might not have, especially schools where they might have a better chance of graduating

The “Realize Your College Potential” campaign targeted hundreds of thousands of students whose SAT scores ranged from modestly above average to the upper echelon. The College Board sent mailers that included a personalized list of potential colleges, fee waivers, and detailed guidance about the application and financial aid processes, along with emails and text messages. (Not all students got every one of these.)

The College Board randomly assigned other groups of students not to get any of the new messages, allowing researchers to make comparisons

Virtually none of these efforts made a meaningful difference, according to a study released this month. Students attended college at similar rates, and those who did went to schools with similar average SAT scores and graduation rates. That included high-achieving, low-income students, the focus of the prior research.

For black and Hispanic students, there was an increase in the average SAT score and graduation rates of the colleges they attended, but that uptick was tiny — just 3 points on the 1600-point SAT, for instance

Oded Gurantz of the University of Missouri, one of the researchers, said he was surprised by the results. But he also noted that students might have benefitted from the guidance in other ways.

“There may be some students who take this information and maybe made a wiser decision for themselves based on unobserved things,” he said. “It could be the student ended up going to the same kind of institution, but they had less stress along the way.”

A spokesperson for the College Board said the organization has drawn lessons from the effort.

“We learned a lot from the packets about how to deliver college planning guidance to students, and we continue to build upon these learnings today, to increase their impact and reach,” said Jerome White. Though they don’t mail the packets anymore, he said, “its components are now embedded across the College Board.”

So can informational ‘nudges’ make a difference at all? It depends

Two other recent efforts, both in Michigan, offer more encouraging evidence that low-cost initiatives can encourage low-income students to attend college.

In one, researchers mailed high-achieving students from low-income families information about a unique full-tuition scholarship at the University of Michigan that they could receive without filling out the FAFSA form. Even though such students were already likely eligible for similar amounts of financial aid, those who received the offer letter were 15 percentage points more likely to enroll in the University of Michigan and much more likely to remain in college two years later. Many of them wouldn’t have attended college at all otherwise.

In another effort, Michigan’s department of education sent 25,000 high school students with above-average ACT scores a letter directing them to a website that offered encouragement and information about college.

The program had no clear effect on college enrollment overall. But for low-income students, getting a letter increased their chances of enrolling in college by 1.4 percentage points (from 76.4% to 77.8%), a modest but statistically significant impact. There were similar improvements in students chances’ of enrolling in a college “matched” to their academic level.

Those benefits, though, faded over time as many of those low-income students left college before graduating. By the second year of college, the effect of the letter had been cut in half and was no longer statistically significant.

This shows that in some cases, the nudges can affect short-run decisions, but not students’ life trajectories — validating concerns that they may not actually be helping students .

“A key concern with light-touch policies is that they may reduce informational and administrative hurdles to the college application process, but not provide students with any lasting improvements in their skills or knowledge,” the study says.

More substantial — and expensive — efforts may make a bigger difference

The research that inspired the College Board’s approach found that a substantial share of high-achieving low-income students were not going to selective colleges they were qualified for, a phenomenon described as “undermatching.” Stanford’s Caroline Hoxby and the University of Virginia’s Sarah Turner showed that a personalized mailer packed with helpful resources, including application-fee waivers, could address this problem, and the researchers encouraged the College Board or another large institution to adopt their approach.

It’s not clear why the College Board didn’t get similar results, even for low-income high achievers.

“We do not think that the College Board’s initiative replicates our experiment, unfortunately,” Hoxby wrote in a brief email to Chalkbeat. “There are so many differences that the College Board’s initiative is not anything close to what we would have put forward. Some of the ‘branding’ elements are highly problematic.”

Hoxby declined to comment further; Turner also declined to comment on the record.

It could be that the College Board was not the right messenger for the college packet, and students discounted the mailings. Slight differences in how the information is packaged and described could affect how students respond to it. The initial flurry of attention to the Hoxby and Turner paper may have also prompted an flood of efforts to help high-achieving, low-income students, making it harder for a single one to stand out.

Meanwhile, some recent research has found that more intensive programs with larger price tags, including state-funded, placebased, and philanthropic scholarship programs, can substantially increase college attendance and completion.

College counselors have also been shown to make a difference. For instance, a Massachusetts initiative to provide intensive in-person college counseling — as well as support during college — to low-income first-generation students increased enrollment in four-year colleges two years after high school by over 20 percentage points, according to a recent study.

Page, for her part, said this is crucial. “If I’m the policymaker with endless resources, I think it’s committing to a proliferation of better college advising supports in schools,” she said. “What we know about college counseling in schools, is it’s left to school counselors who have caseloads that are too large.”