It is hard to not see a reference to ChatGPT in musings and reflections on higher education today. The A.I. software that can write essays, responding to most prompts with ease, is worrying many faculty who fear students will use this technology to cheat. Students can, and probably will, use this software. What they use it for is the question. Faculty can play a big role in what students use ChatGPT for and how students use ChatGPT.
Here at Oregon State University the Office of Academic Affairs has pulled together a group of key members of the faculty who with staff from the Center for Teaching and Learning, Academic Technologies and UIT, and Ecampus, will provide pedagogical guidance in the weeks ahead. In the meantime, two recent posts provide a quick story so far. OSU’s Dr. Inara Scott (College of Business) wrote a blog with some initial recommendations for faculty and compliments a recent Chronicle of Higher Education piece which provides some key sources for further information including webinars.
The fact that with ChatGPT there are “appropriate” and even beneficial, exciting, and useful uses is lost on many. It is a pedagogical affordance that many faculty are taking advantage of and should as AI is just going to get better. I came across one great example I wanted to share. Dr. Morton Ann Gernsbacher is the Vilas Research Professor and the Sir Frederic C. Bartlett Professor of Psychology at the University of Wisconsin. She provides a great assignment on how to face ChatGPT head on and provides key insights into how to think about ChatGPT. She agreed to allow me to share her recent thoughts from social media. Make sure you check out her pdf with links too.
Hi everyone. I’ve developed a ChatGPT assignment that meets my pedagogical goals, and I thought I’d share my assignment with others if they’d like to use it. As with all my assignments, all the materials are open-access, so feel free to use whatever you’d like!
My goals for the assignment were the following:
1) Require students to become familiar with ChatGPT.
2) Require students to experience that ChatGPT can be erratically accurate.
3) Require students to become familiar with a ChatGPT detector.
4) Require students to commit to informing me if they use ChatGPT for their work in my course.
I required students to become familiar with ChatGPT, my first goal, for both equity (I don’t want some students to be aware of it but not others) and because my course is upper-division and titled “Psychological Effects of the Internet” (although I plan to use a version of this assignment in all my courses, including Basic Stats and Research Methods).
I achieved this first goal by excerpting two recent popular press articles and a collection of recent Tweets about ChatGPT showing both its power and its pitfalls. For the Tweets, I tried to spotlight at least one celebrity (Flavor Flav!).
Also, because my ChatGPT assignment occurs in my course’s first unit, during which students have been learning about previous technological moral panics (some centuries old, e.g., printed novels, recorded music in movie theaters, hand calculators, even ballpoint pens), students in my course were assigned two additional brief articles about moral panics over technology.
Students were then assigned two recent Tweets (well, one Tweet and one Mastodon post) written by educators recommending that the best way to avoid a moral panic about ChatGPT is to teach students how to use it critically (aka: apply critical thinking), which is what I aspired to do. Therefore, for my second goal, that of requiring students to experience that ChatGPT can be erratically accurate, I constructed six questions about my university (University of Wisconsin-Madison). I had pre-tested these questions to feel confident that ChatGPT’s answers would be somewhat correct but also incorrect. I required students to use their critical thinking to evaluate the responses ChatGPT provided, and so far, that is working out well. Not every ChatGPT response is 100% inaccurate, and even if some would be 100% accurate, students need to use critical thinking to distinguish the accuracies from the inaccuracies.
For my second goal, that of requiring students to become familiar with a ChatGPT detector, I again did that for equity (I again didn’t want some students to be aware of it but not others). Students were required to copy/paste the text of one of their own previous assignments into the ChatGPT detector and to copy/paste the text of a ChatGPT-generated assignment into the detector.
So far, the ChatGPT detector activity is working well, with the detector typically considering the students’ assignments as “99.9% Real” and the ChatGPT-generated assignment as “99.9% Fake.” However, along the way I learned an interesting quirk about the detector. What I found was if I copied/pasted into the ChatGPT detector the exact same text from a PDF versus a Word document, the ChatGPT detector treated it differently! The content was verbatim the same. Literally, every single word was the same. But copying from the PDF caused funky line breaks (as I think we’ve all experienced when copying/pasting from a PDF). And those funky line breaks somehow caused the detector to think the content was “Real” rather than “Fake” (aka: ChatGPT created).
Copying/pasting from a Word doc (or I’d guess a Google doc, Pages file, or anything without hard line breaks) seemed to do the trick; the detector identified the ChatGPT-created text as “Fake.”
I tested this multiple times and even asked someone else to test it. Every time, the funky line breaks caused the detector to claim the text was Real; the normal line breaks caused the detector to accurately claim the text was Fake.
I was stumped by this and consulted a colleague who does a lot of machine learning (like the ChatGPT AI bot is — and who has *almost* been as obsessed about ChatGPT as I have) was equally stumped about this. I think the bottom line is that the detector isn’t 100% foolproof, and that’s good for instructors to know if they’re planning to use it for grading purposes.
Lastly, I required students to commit to the following statement: “I know that in this course I can use ChatGPT, but I must always apply critical thinking to anything ChatGPT tells me AND I must always make a Gradebook Comment (not a Discussion Board post, but a Gradebook Comment) telling the instructor and TAs whenever I have used ChatGPT and how I have used it.”
The entire assignment is attached in a PDF with links. As I mentioned before, please feel free to use whatever parts you’d like to use.
Over the past two weeks, I have learned a lot about ChatGPT. If anyone would like to engage with me more about this topic, please feel free to email me at magernsb@wisc.edu. Thank you so much!
Thank YOU, Dr. Gernsbacher.
Leave a Reply