Heavy AI Reliance Lowers Test Scores Despite Higher Satisfaction

The AI Learning Paradox

Students who lean heavily on generative AI tools during problem-solving score lower on tests even as they report higher satisfaction with their learning experience, according to new research from Sungkyunkwan University in South Korea.

The study, which examined 88 university students working through high-school-level academic material, reveals a troubling disconnect between perceived and actual learning outcomes when AI becomes a crutch rather than a catalyst.

How the Experiment Worked

Researchers divided the learning process into three distinct phases: concept understanding (reading new material), problem solving (applying knowledge), and result review (checking answers). Students were randomly assigned to use GPT-4o at different stages, allowing the team to isolate where AI assistance proved most beneficial.

The findings showed clear advantages for strategic AI use. Students who employed the tool during problem-solving outperformed those who used it only during initial concept review. The AI-generated hints and summaries reduced cognitive load and eliminated unnecessary mental friction, boosting efficiency.

But the benefit disappeared for high-dependency users—those who habitually relied on AI across their studies. Within the problem-solving group, students with stronger AI dependency patterns posted lower test scores. The research team attributed this to passive acceptance of AI-provided answers without the critical thinking required to internalize knowledge.

The Illusion of Mastery

The most striking result emerged in self-assessment data. High-dependency students reported substantially higher perceived learning and satisfaction levels despite their weaker objective performance. Researchers interpret this as a "learning illusion"—the AI's polished explanations and accessible formatting create a false sense of comprehension that doesn't translate to independent problem-solving ability.

This gap between confidence and competence represents a significant risk as generative AI tools proliferate in educational settings without corresponding guidance on effective use.

Why It Matters

This research moves beyond binary debates about allowing or banning AI in education. It demonstrates that how students use these tools matters more than whether they use them at all. The findings suggest educators need frameworks that encourage active engagement with AI outputs rather than passive consumption—teaching students to question, verify, and synthesize rather than simply accept generated content. Without such guardrails, institutions risk producing learners who feel knowledgeable but lack the deep understanding required for complex reasoning.

Toward Smarter Implementation

Professor Changjun Lee, one of the study's authors, stressed that generative AI functions as a powerful efficiency tool but warned against uncritical reliance. He called for educational guidelines that develop students' independent questioning and critical verification skills, positioning AI as a learning aid rather than a replacement for cognitive effort.

The research, supported by South Korea's Ministry of Education and National Research Foundation, was first reported by Sungkyunkwan University and conducted by Somi Joo, Changjun Lee, and Daeho Lee from the Department of Artificial Intelligence Convergence.