Ludwig-Maximilians-Universität München

A physics student solves a difficult problem with AI assistance, submits a correct solution and receives a good grade. From the outside, this looks like successful learning. But has the student actually understood the physics? Can they explain the underlying principles without assistance? Can they recognize an incorrect AI-generated explanation? Can they transfer their understanding to an unfamiliar problem?

These questions reveal a fundamental challenge for higher education: AI can improve students' performance without necessarily improving their learning.

The Performance-Learning Paradox

This distinction is increasingly supported by empirical research.

In a recent systematic review and meta-analysis by Boolzen and colleagues of generative AI in STEM education, covering 85 studies, the conventional meta-analysis estimated substantial positive effects. However, a publication-bias-adjusted analysis yielded an estimated overall effect close to zero.. The findings also highlighted the importance of how AI is integrated into learning activities.

A separate methodological audit by Fütterer and colleagues of early ChatGPT meta-analyses revealed another concern. Among 69 classifiable primary studies, 79.7 percent assessed performance exclusively, while only 11.6 percent assessed learning exclusively. Another 8.7 percent measured both.

In other words, much of the evidence presented as demonstrating that AI improves learning primarily measures task performance rather than independently established learning gains. This does not mean that AI cannot support learning. It means that we must distinguish between what students accomplish with AI and what they become capable of doing because of learning with AI.

Augmentation, Substitution and Epistemic Agency

AI can provide explanations, generate alternative representations, identify mistakes and offer individualized feedback. These capabilities can create valuable opportunities for learning. However, AI can also replace precisely those cognitive activities through which understanding develops.

Consider two students working on the same physics problem. One asks AI for a complete solution and studies the explanation. The other first develops an approach, uses AI to challenge assumptions, compares alternative solutions and revises their reasoning.

Both may produce correct answers. Yet the learning opportunities are fundamentally different. The central question is therefore not how much assistance AI provides, but which cognitive and regulatory responsibilities remain with the learner. This brings us to epistemic agency, the capacity to formulate questions, evaluate evidence, construct explanations and take responsibility for knowledge-related decisions.

Preserving epistemic agency does not mean that students must perform every task independently. AI can productively take over routine operations, freeing cognitive resources for deeper reasoning. But when it replaces the very activities students are supposed to learn, assistance becomes substitution.

Turning this principle into effective learning environments requires deliberate decisions about when AI should assist and when learners must retain responsibility.

“Ultimately, the success of AI in education should not be measured by how much thinking it can do for students, but by how much better students learn to think with - and eventually without - it.”

This is the central idea behind AIRIS (AI-Augmented Inquiry and Regulation in Hybrid Systems), a theoretical framework we are developing together with international colleagues to guide the allocation of cognitive and regulatory responsibilities in human– AI collaboration. Rather than simply providing answers, AI can prompt learners to anticipate outcomes, question explanations, reflect on their reasoning, integrate new insights and synthesize their own understanding.

The aim is to design interactions in which AI strengthens epistemic agency instead of gradually replacing it.

Rethinking Assessment

If AI can help students produce increasingly sophisticated outputs, traditional assignments become less informative about what students actually know. A polished essay, a correct mathematical solution or a well-structured laboratory report may ref lect the capabilities of the student, the AI system or their interaction.

Universities must therefore distinguish between assessing the quality of AI-supported products and assessing the competencies students have developed. This does not require abandoning AI-assisted assignments. Instead, assessment should combine authentic AI-supported work with opportunities to demonstrate independent understanding.

Students might defend their reasoning orally, evaluate deliberately flawed AI-generated explanations, transfer concepts to unfamiliar problems or solve related tasks without assistance.

Three questions should guide the evaluation of AIsupported learning

- What can students understand and accomplish independently after learning with AI?

- Does AI strengthen their ability to reason critically, evaluate evidence and solve unfamiliar problems?

- How should AI assistance be adjusted over time to promote independence rather than dependence?

Answering these questions requires collaboration among educators, learning scientists, technology developers and university leaders.

The Real Transformation of Higher Education

The future of higher education will not be determined by how quickly universities adopt AI tools. It will depend on whether they deliberately shape the learning processes that AI supports. This requires a shift in priorities, from technological availability to educational design, from immediate performance to lasting competence and from automated answers to independent judgment.

AI offers extraordinary opportunities for personalized feedback, scientific inquiry and deeper understanding. But these opportunities will not materialize automatically.

Ultimately, the success of AI in education should not be measured by how much thinking it can do for students, but by how much better students learn to think with and without it.