By Joseph J. Slade, OSU School of Psychological Science
Artificial intelligence (AI) tools like ChatGPT have exploded onto the educational scene, bringing both excitement and concern. Faculty and instructional designers find themselves grappling with a burning question: Does student use of AI enhance learning outcomes, or does it hinder deep learning and critical thinking? Given that AI can generate answers and provide feedback in seconds, it is easy to see the appeal. But if students rely on AI to complete their schoolwork, they might skip the mental heavy lifting that leads to true understanding. Thus far, research findings paint a complicated picture: while AI can boost certain learning outcomes, it may also encourage shortcuts that undermine critical thinking. This post will provide an overview of some of the most recent research on the subject and discuss how it might be possible to harness AI’s benefits without sacrificing deep learning.
Early evidence of AI’s academic benefits
It is still early days for experimental research on how AI tools impact student performance, and unsurprisingly, the findings are mixed. On the positive side, a growing number of studies indicate that AI can improve academic outcomes. A recent meta-analysis of 51 studies found that using ChatGPT had a large positive effect on student learning performance (g = 0.867), a moderately positive effect on how much students perceived that they learned (g = 0.456) as well as a moderate positive effect on higher-order thinking skills (g = 0.457) (Wang & Fan, 2025). In other words, students who incorporated ChatGPT into their learning tended to perform better on assessments, report more learning, and show gains in advanced thinking tasks compared to those who did not use AI. The authors themselves did caution, however, that sample sizes for improvements in perceived learning and higher-order thinking were small enough to warrant follow-up studies. This review only included experimental or quasi-experimental designs and participants included K-12 and college students.
These broad gains were echoed by a December 2024 review which analyzed 69 studies to conclude that ChatGPT generally improves students’ academic performance and even their motivation, while also enhancing “higher-order thinking” tendencies (Deng et al., 2024). Notably, the same review observed that AI assistance tends to reduce students’ mental effort on tasks (making learning feel easier) without significantly affecting their self-confidence as learners. Most of the studies included in this review were at the university level (84.06%) with the remainder being K-12 students. Again, the authors cautioned that many of the reviewed studies were insufficiently powered and may have suffered from methodological issues (for example, some studies allowed participants to make use of ChatGPT during post-intervention assessments).
The downside of overreliance on AI
But not all studies agree that AI is beneficial to learning. Some research finds little to no improvement from AI use, and a few studies even suggest it can hinder learning under certain conditions.
A December 2024 study in the British Journal of Education Technology examined how generative AI affects student learning, motivation, and writing processes. In a lab experiment with 117 Chinese students writing an essay in English, participants were assigned to one of four groups: using ChatGPT, working with a human writing coach, using a checklist-based writing toolkit, or receiving no extra help (Fan et al., 2025). While the ChatGPT group produced the greatest essay quality (even surpassing that of the group using human tutors) participants did not learn more about the material, feel more motivated, or sustain greater interest compared to other groups. In fact, students with only the checklist showed the highest engagement and time spent reviewing their work. AI- and human-assisted students consulted source materials less, focused more heavily on their helper rather than the readings, and the AI group often directly copy-and-pasted generated text. Researchers concluded that ChatGPT use risked fostering “metacognitive laziness,” with students offloading critical thinking and problem-solving to the bot instead of actively engaging in any synthesis and analysis.
Similar results were found in a field experiment with nearly 1,000 high school math students (Bastani et al., 2025). Researchers compared two GPT-4–powered tutors: a standard ChatGPT-style interface (“GPT Base”) and a version with prompts designed to support learning (“GPT Tutor”). While both improved problem-solving performance during use (by 48% and 127%, respectively), removing AI access later revealed a downside; students who had used GPT Base scored 17% lower than peers with no AI, suggesting that an overreliance on the tool may have hindered skill retention. These negative effects were largely avoided with the safeguarded GPT Tutor, highlighting the possibility that AI can indeed be used to improve learning and performance.
Can AI help my students?
Some educators have had great success with AI-driven activities, such as having students debate a topic with ChatGPT or critique an AI-generated argument. Undergraduate students in one International Relations course used ChatGPT as a tool during debate exercises. The instructors used ChatGPT to provide instant feedback and counterarguments, prompting students to strengthen their reasoning. This study found significant improvements in the students’ critical thinking and argumentation skills compared to a control group who didn’t use AI (de la Puente et al., 2024).
Thus far, AI’s impact on learning and critical thinking remains uncertain and is still evolving, with research suggesting that its effects may depend less on the technology itself and more on how it is used. When employed as a guided practice tool or tutor, AI can support deeper understanding and help build skills such as argumentation. However, when relied upon as an on-demand answer generator, it risks promoting superficial learning and disengagement, raising the question of whether students can still learn to analyze, evaluate, and create on their own. The mixed results of recent studies highlight this double-edged nature, underscoring that thoughtful integration is key to ensuring AI serves as a pathway to deeper learning rather than a barrier to it.
References
Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. https://doi.org/10.1073/pnas.2422633122
de la Puente, M., Torres, J., Troncoso, A. L. B., Meza, Y. Y. H., & Carrascal, J. X. M. (2024). Investigating the use of ChatGPT as a tool for enhancing critical thinking and argumentation skills in international relations debates among undergraduate students. Smart Learning Environments, 11(1), 55. https://doi.org/10.1186/s40561-024-00347-0
Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2024). Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies. Computers & Education, 227, 105224. https://doi.org/10.1016/j.compedu.2024.105224
Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025). Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology, 56(2), 489-530. https://doi.org/10.1111/bjet.13544
Wang, J., & Fan, W. (2025). The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: Insights from a meta-analysis. Humanities and Social Sciences Communications, 12, 621. https://doi.org/10.1057/s41599-025-04787-y
About the author: Joseph J. Slade is a psychology Ph.D. student at Oregon State University studying the intersection of AI, teaching, and learning. He serves as the director of Project FAILSafe (Fostering AI Learning Safely), a collaboration between the School of Psychological Science and the School of Computer Science and Electrical Engineering that is developing AI educational tools and testing their efficacy through controlled experiments and classroom applications. His work focuses on identifying how educators can make best use of large language models to improve student engagement and learning outcomes.
Top image generated with Microsoft Copilot
Leave a Reply