Did you know search engines like Google are changing the way you see news? Keep Chalkbeat’s valuable education coverage at the top of your feeds by clicking here and checking the box next to our name.
Most AI tools don’t help students learn. Education-specific AI tools aren’t necessarily better than general purpose chatbots. And even tools that help teachers save time can have unintended consequences for students.
Those are the conclusions from a review of 52 studies on the use of artificial intelligence in educational settings released Tuesday by the AI Hub for Education, part of Stanford University’s SCALE Initiative, which studies which classroom interventions and tools actually work.
The authors of the report aren’t throwing in the towel on AI in education. There is still very little research on AI in K-12 settings in the United States — many of the studies come from other countries or involve college students — and the technology has potential to provide personalized assistance for students and remove some of the tedium from teachers’ jobs.
That potential is worth developing, said Susanna Loeb, faculty director at the SCALE Initiative and an author of the report.
AI tools can be “quite powerful,” Loeb said, to differentiate lessons for students at different levels or support students learning a new language. “You just have to use them thoughtfully and track how they are doing and whether they’re really benefiting students.”
But teachers and district leaders need to proceed thoughtfully. If they don’t have the capacity to do so, it’s better not to use AI at all, Loeb said.
“If you are a district that doesn’t want to think about it and you don’t want to design your classwork and homework and school day to leverage these tools well, I wouldn’t use them,” Loeb said. “On their own without thought, they have some negative effects on learning.”
The new research review comes as school districts are navigating competing pressures to adopt AI rapidly and to limit technology in the classroom. Google caused an uproar this month when it said that school districts that ban AI will not be able to use Google Search. The American Federation of Teachers, meanwhile, has embraced training teachers to use AI while pressing major tech companies to agree to guardrails.
This is the AI Hub’s second AI research review, and it incorporates 32 new studies. The included studies, selected from hundreds in the field, are either randomized controlled trials or have a quasi-experimental design and provide strong causal evidence. The goal is to eventually have a robust research repository that teachers and districts can use to inform their decisions about whether and how to use AI.
Collectively, the studies suggest students don’t learn much when they use AI to produce essays or solve problems. When those students later do work without AI, they do worse than other students. AI shows more promise when students use it to explain concepts and help them practice, but even then the results are mixed or don’t show much benefit over traditional study methods.
The new studies include some positive findings. For example, AI tools that provide interactive formats and adaptive practice questions may improve student learning more than static digital textbooks. And students sometimes experience AI-generated feedback as clearer and more motivating than feedback from teachers. But even where students reported positive experiences, the evidence of learning gains was mixed.
The studies seem to show more benefit for teachers than for students. Teachers who used an AI tool to do the first pass on grading assignments provided more and more detailed feedback in one study. And teachers who used AI progress-tracking tools saw students complete more assignments, though students’ grades on those assignments weren’t necessarily better.
Studies also found that teachers could save time using AI to generate lesson plans without reducing the quality of those lessons. However, students who believed a lesson was AI-generated reported being less motivated. And in one study from Turkey, students did worse when teachers who already were lower-performing used AI to generate lesson plans.
The research review is a synthesis of findings from high-quality studies, not a meta-analysis, which would involve analyzing multiple studies and using their collective findings to reach a stronger conclusion.
AI in education is not just one thing — there are too many different tools being used in too many different ways to pile studies together and reach strong conclusions.
That was one of the flaws with a number of previous meta-analyses that found positive outcomes for AI in education. A forensic audit found more serious flaws as well, such as hallucinated references and statistics that didn’t make sense, according to The Hechinger Report.
Another challenge with studying AI in education, particularly with younger students: They often don’t want to use the tools.
Erica Meltzer is Chalkbeat’s national editor based in Colorado. Contact Erica at emeltzer@chalkbeat.org.
