{"id":6000,"date":"2026-04-09T07:59:33","date_gmt":"2026-04-09T15:59:33","guid":{"rendered":"https:\/\/blogs.oregonstate.edu\/osuteaching\/?p=6000"},"modified":"2026-04-09T12:11:22","modified_gmt":"2026-04-09T20:11:22","slug":"intentional-ai-spotlight-jonathan-kalodimos-on-evaluating-learning-when-students-cant-hide-behind-the-text","status":"publish","type":"post","link":"https:\/\/blogs.oregonstate.edu\/osuteaching\/2026\/04\/09\/intentional-ai-spotlight-jonathan-kalodimos-on-evaluating-learning-when-students-cant-hide-behind-the-text\/","title":{"rendered":"Intentional AI Spotlight: Jonathan Kalodimos on assessing learning when students can&#8217;t hide behind the text"},"content":{"rendered":"\n<p>By Demian Hommel, CTL AI in Teaching and Learning Fellow in partnership with the AI Literacy Center<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><a href=\"https:\/\/i0.wp.com\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/1441\/files\/2026\/03\/Stories_of_AI_OSU_3to1_hero_text_D73F09-Copilot-11mar26.png?ssl=1\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"1536\" height=\"512\" src=\"https:\/\/i0.wp.com\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/1441\/files\/2026\/03\/Stories_of_AI_OSU_3to1_hero_text_D73F09-Copilot-11mar26.png?resize=1536%2C512&#038;ssl=1\" alt=\"Stories of AI at OSU\" class=\"wp-image-5901\" srcset=\"https:\/\/i0.wp.com\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/1441\/files\/2026\/03\/Stories_of_AI_OSU_3to1_hero_text_D73F09-Copilot-11mar26.png?w=1536&amp;ssl=1 1536w, https:\/\/i0.wp.com\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/1441\/files\/2026\/03\/Stories_of_AI_OSU_3to1_hero_text_D73F09-Copilot-11mar26.png?resize=300%2C100&amp;ssl=1 300w, https:\/\/i0.wp.com\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/1441\/files\/2026\/03\/Stories_of_AI_OSU_3to1_hero_text_D73F09-Copilot-11mar26.png?resize=1024%2C341&amp;ssl=1 1024w, https:\/\/i0.wp.com\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/1441\/files\/2026\/03\/Stories_of_AI_OSU_3to1_hero_text_D73F09-Copilot-11mar26.png?resize=768%2C256&amp;ssl=1 768w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/a><\/figure>\n\n\n\n<p>As part of the Intentional AI at OSU series, I sat down with Jonathan Kalodimos, an associate professor of finance and Harley &amp; Brigitte Smith Fellow. Jon&#8217;s background includes a tenure as a financial economist at the U.S. Securities and Exchange Commission, and he brings that same empirical rigor to a problem now facing every department on campus: generative AI has made written work unreliable as a signal of what students actually know. His response is to build a system that makes oral evaluation\u2014where you can hear students think\u2014feasible at scale.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The challenge: Seeing what students actually know<\/h2>\n\n\n\n<p>In a traditional classroom, written assignments have been the primary window into student understanding. Generative AI has clouded that window. For Jon, the challenge goes deeper than academic integrity:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Writing has become opaque: A well-prompted chatbot can produce a polished essay or financial analysis indistinguishable from strong student work. The signals faculty have relied on for decades\u2014clarity of argument, depth of analysis\u2014are now easily manufactured. Detection software is unreliable and shifts the focus from learning to policing.<\/li>\n\n\n\n<li>Oral evaluation reveals reasoning: When a student presents orally, they have to explain, connect, and defend ideas. You get a direct signal of what they actually understand. But grading presentations is one of the most time-intensive things faculty do, which keeps most instructors from assigning them at the frequency students need.<\/li>\n\n\n\n<li>The gap is structural: Faculty need a system that handles the labor-intensive parts of evaluating oral work\u2014extracting evidence, mapping it to criteria\u2014so they can focus their limited time on exercising judgment.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">The innovation: A faculty-AI partnership for oral assessment<\/h2>\n\n\n\n<p>Jon has built a system at OSU that processes student video presentations and produces evidence-based evaluation drafts for faculty review\u2014a partnership that makes better assessment sustainable.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Evidence extraction: The system goes through every word and image a student said and every piece of evidence in their slides, then pulls out the specific evidence relevant to each rubric criterion. The output is a draft showing the instructor exactly what the student demonstrated, with defined evidence extracted from the presentation.<\/li>\n\n\n\n<li>Cognitive classification using Bloom&#8217;s taxonomy: Beyond collecting evidence, the system maps what it finds against Bloom&#8217;s taxonomy. Did the student merely recall a definition or explain a concept in their own words? Apply a method or analyze trade-offs between approaches? This gives faculty a structured starting point for evaluating the depth of student thinking, which they can adjust based on their own professional judgment.<\/li>\n\n\n\n<li>Domain-agnostic by design: Because the system evaluates what students say and write on their slides rather than discipline-specific structures, it works across disciplines. The same framework applies whether a finance student is defending a valuation, a geography student is presenting research, or an engineering student is walking through a technical design. What changes is the rubric, not the system.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Reflection: AI as a partner in better assessment<\/h2>\n\n\n\n<p>Most of the time faculty spend evaluating oral work goes to gathering evidence, not making judgments. Jon&#8217;s system redirects that balance: if AI can reliably surface what a student said and map it to your criteria, you spend your time on the decisions that require your expertise.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Key advice for faculty<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Start with oral, even small: As written work becomes easier for AI to generate, consider adding even short oral components\u2014a three-minute video explanation of a concept gives you a window into student reasoning that a polished essay no longer provides.<\/li>\n\n\n\n<li>Write criteria around what&#8217;s observable: Whether or not you use AI-assisted evaluation, rubrics built around observable behaviors produce better feedback. &#8220;Student explains the relationship between X and Y using a specific example&#8221; is actionable. &#8220;Student demonstrates deep understanding&#8221; is not.<\/li>\n\n\n\n<li>Treat AI outputs as drafts: AI evaluation systems tend to be generous; they treat any mention of the right concepts as evidence of comprehension. Faculty review is the mechanism that maintains rigor.<\/li>\n\n\n\n<li>Think beyond your own classroom: The assessment challenges created by generative AI aren&#8217;t unique to any single discipline. Faculty who engage with these approaches now are helping shape what AI-assisted education looks like across the institution.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<figure class=\"wp-block-image alignleft size-full\"><a href=\"https:\/\/i0.wp.com\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/1441\/files\/2025\/09\/Screenshot-2025-09-29-at-14-16-15-Demian-Hommel.jpg.webp-WEBP-Image-600-%C3%97-600-pixels.jpeg?ssl=1\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"220\" height=\"220\" src=\"https:\/\/i0.wp.com\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/1441\/files\/2025\/09\/Screenshot-2025-09-29-at-14-16-15-Demian-Hommel.jpg.webp-WEBP-Image-600-%C3%97-600-pixels.jpeg?resize=220%2C220&#038;ssl=1\" alt=\"Demian Hommel.\" class=\"wp-image-5278\" srcset=\"https:\/\/i0.wp.com\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/1441\/files\/2025\/09\/Screenshot-2025-09-29-at-14-16-15-Demian-Hommel.jpg.webp-WEBP-Image-600-%C3%97-600-pixels.jpeg?w=220&amp;ssl=1 220w, https:\/\/i0.wp.com\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/1441\/files\/2025\/09\/Screenshot-2025-09-29-at-14-16-15-Demian-Hommel.jpg.webp-WEBP-Image-600-%C3%97-600-pixels.jpeg?resize=150%2C150&amp;ssl=1 150w\" sizes=\"auto, (max-width: 220px) 100vw, 220px\" \/><\/a><\/figure>\n\n\n\n<p><em>About the Author:<\/em> <strong>Demian Hommel<\/strong> is a professor of geography and environmental science in the College of Earth, Ocean, and Atmospheric Sciences and is an AI in Teaching and Learning Fellow with the OSU Center for Teaching and Learning. When he isn\u2019t exploring the societal and environmental impacts of AI, you can find him DJing under the alias <em>Dr. Gonzo<\/em> or trying to graft citrus trees in his greenhouse.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<p>Top image generated with Microsoft Copilot.<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>By Demian Hommel, CTL AI in Teaching and Learning Fellow in partnership with the AI Literacy Center As part of the Intentional AI at OSU series, I sat down with Jonathan Kalodimos, an associate professor of finance and Harley &amp; Brigitte Smith Fellow. Jon&#8217;s background includes a tenure as a financial economist at the U.S. [&hellip;]<\/p>\n","protected":false},"author":3089,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[448432,448597,448504,448344,448596],"class_list":["post-6000","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-ai","tag-ai-spotlights","tag-blooms-taxonomy","tag-rubric","tag-stories-of-ai-osu"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blogs.oregonstate.edu\/osuteaching\/wp-json\/wp\/v2\/posts\/6000","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.oregonstate.edu\/osuteaching\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.oregonstate.edu\/osuteaching\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/osuteaching\/wp-json\/wp\/v2\/users\/3089"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/osuteaching\/wp-json\/wp\/v2\/comments?post=6000"}],"version-history":[{"count":5,"href":"https:\/\/blogs.oregonstate.edu\/osuteaching\/wp-json\/wp\/v2\/posts\/6000\/revisions"}],"predecessor-version":[{"id":6013,"href":"https:\/\/blogs.oregonstate.edu\/osuteaching\/wp-json\/wp\/v2\/posts\/6000\/revisions\/6013"}],"wp:attachment":[{"href":"https:\/\/blogs.oregonstate.edu\/osuteaching\/wp-json\/wp\/v2\/media?parent=6000"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/osuteaching\/wp-json\/wp\/v2\/categories?post=6000"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/osuteaching\/wp-json\/wp\/v2\/tags?post=6000"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}