AI Tutoring Tools Face a Student Engagement Problem

Artificial intelligence tutoring tools are hitting an unexpected barrier: students simply aren't using them, even when the technology is readily available and designed to help.

A two-year study of Khan Academy's AI-powered chatbot Khanmigo found that middle school students in Tennessee rarely engaged with the tool in productive ways. The research, conducted by University of Toronto's Philip Oreopoulos and colleagues, tracked low-performing students across 18 middle schools who were randomly assigned to use Khan Academy during the school day.

The findings reveal a fundamental challenge for educational AI. Students used Khanmigo on only about one-third of the days they worked in Khan Academy. When they did interact with the chatbot, many sent off-topic messages or attempted to extract correct answers rather than learning through guided problem-solving.

What the research found

The study compared students using Khan Academy with peers who used a mix of other digital math programs and small-group teacher instruction. By the second year, Khan Academy students showed faster math gains than the comparison group—a positive signal for the platform itself.

However, researchers concluded these benefits didn't stem from the AI component. "Access was nearly universal but engagement was thin," the study noted. "It may be that human attention is the key ingredient for realizing the full benefits of personalized learning."

Khanmigo is programmed to help students without simply providing answers, but it requires students to actively seek assistance. The passive availability wasn't enough to drive meaningful engagement.

A second study tests forced interaction

In a separate study released the same week, Oreopoulos and colleagues tried a different approach in the same Tennessee district. They designed an online math program where an AI chatbot appeared automatically without student prompting. Students could only advance after answering questions correctly.

This more structured approach showed modest promise. Students moved more slowly through problems but answered more accurately. A week later, when tested on similar problems without AI assistance, they performed slightly better than a control group.

Why it matters

The research highlights a critical gap between AI capability and real-world educational impact. As schools invest in AI tutoring systems, the technology's theoretical benefits mean little if students won't engage with it productively. The findings suggest that simply providing access to sophisticated AI tools isn't sufficient—schools may need to design tightly controlled environments that require student interaction, raising questions about scalability and student autonomy.

"The same technology that can be used as a good tutor, can also be used to make your life easier. But making your life easier equals less effort, less learning," Oreopoulos said.

Since the study period, Khan Academy has redesigned its interface to better integrate Khanmigo into the learning experience. Founder Sal Khan acknowledged the AI tutor had been a "non-event" for most students and wrote that the platform needed to make "productive struggle harder to sidestep."

The challenge extends beyond Khan Academy. A recent research summary concluded that "when given AI tutors, many students do not engage"—a pattern emerging across multiple studies of educational AI tools.

These findings were first reported by Chalkbeat.