Category Archives: Teaching

Media Literacy in the Age of AI, Part II: A Review of Verified

Posted on by | Leave a reply

For the first part of this post, please see Media Literacy in the Age of AI, Part I: “You Will Need to Check It All.”

Just how, exactly, we’re supposed to follow Ethan Mollick’s caution to “check it all” happens to be the subject of a lively, forthcoming collaboration from two education researchers who have been following the intersection of new media and misinformation for decades.

In Verified: How to Think Straight, Get Duped Less, and Make Better Decisions about What to Believe Online (University of Chicago Press, November 2023), Mike Caulfield and Sam Wineburg provide a kind of user’s manual to the modern internet. The authors’ central concern is that students—and, by extension, their teachers—have been going about the process of verifying online claims and sources all wrong—usually by applying the same rhetorical skills activated in reading a deep-dive on Elon Musk or Yevgeny Prigozhin, to borrow from last month’s headlines. Academic readers, that is, traditionally keep their attention fixed on the text—applying comprehension strategies such as prior knowledge, persisting through moments of confusion, and analyzing the narrative and its various claims about technological innovation or armed rebellion in discipline-specific ways.

The Problem with Checklists

Now, anyone who has tried to hold a dialogue on more than a few pages of assigned reading at the college level knows that sustained focus and critical thinking can be challenging, even for experienced readers. (A majority of high school seniors are not prepared for reading in college, according to 2019 data.) And so instructors, partnering with librarians, have long championed checklists as one antidote to passive consumption, first among them the CRAAP test, which stands for currency, relevance, authority, accuracy, and purpose. (Flashbacks to English 101, anyone?) The problem with checklists, argue Caulfield and Wineburg, is that in today’s media landscape—awash in questionable sources—they’re a waste of time. Such routines might easily keep a reader focused on critically evaluating “gameable signals of credibility” such as functional hyperlinks, a well-designed homepage, airtight prose, digital badges, and other supposedly telling markers of authority that can be manufactured with minimal effort or purchased at little expense, right down to the blue checkmark made infamous by Musk’s platform-formerly-known-as-Twitter.

Three Contexts for Lateral Reading

One of the delights in reading Verified is drawing back the curtains on a parade of little-known hoaxes, rumors, actors, and half-truths at work in the shadows of the information age—ranging from a sugar industry front group posing as a scientific think tank to headlines in mid-2022 warning that clouds of “palm-sized flying spiders” were about to descend on the East Coast. In the face of such wild ideas, Caulfield and Wineburg offer a helpful, three-point heuristic for navigating the web—and a sharp rejoinder to the source-specific checklists of the early aughts. (You will have to read the book to fact-check the spider story, or as the authors encourage, you can do it yourself after reading, say, the first chapter!) “The first task when confronted with the unfamiliar is not analysis. It is the gathering of context” (p. 10). More specifically:

  • The context of the source — What’s the reputation of the source of information that you arrive at, whether through a social feed, a shared link, or a Google search result?
  • The context of the claim — What have others said about the claim? If it’s a story, what’s the larger story? If a statistic, what’s the larger context?
  • Finally, the context of you — What is your level of expertise in the area? What is your interest in the claim? What makes such a claim or source compelling to you, and what could change that?
“The Three Contexts” from Verified (2023)

At a regional conference of librarians in May, Wineburg shared video clips from his scenario-based research, juxtaposing student sleuths with professional fact checkers. His conclusion? By simply trying to gather the necessary context, learners with supposedly low media literacy can be quickly transformed into “strong critical thinkers, without any additional training in logic or analysis” (Caulfield and Wineburg, p. 10). What does this look like in practice? Wineburg describes a shift from “vertical” to “lateral reading” or “using the web to read the web” (p. 81). To investigate a source like a pro, readers must first leave the source, often by opening new browser tabs, running nuanced searches about its contents, and pausing to reflect on the results. Again, such findings hold significant implications for how we train students in verification and, more broadly, in media literacy. Successful information gathering, in other words, depends not only on keywords and critical perspective but also on the ability to engage in metacognitive conversations with the web and its architecture. Or, channeling our eight-legged friends again: “If you wanted to understand how spiders catch their prey, you wouldn’t just look at a single strand” (p. 87).

SIFT graphic by Mike Caulfield with icons for stop, investigate the source, find better coverage, and trace claims, quotes, and media to the original context.

Image 2: Mike Caulfield’s “four moves”

Reconstructing Context

Much of Verified is devoted to unpacking how to gain such perspective while also building self-awareness of our relationships with the information we seek. As a companion to Wineburg’s research on lateral reading, Caulfield has refined a series of higher-order tasks for vetting sources called SIFT, or “The Four Moves” (see Image 2). By (1) Stopping to take a breath and get a look around, (2) Investigating the source and its reputation, (3) Finding better sources of journalism or research, and (4) Tracing surprising claims or other rhetorical artifacts back to their origins, readers can more quickly make decisions about how to manage their time online. You can learn more about the why behind “reconstructing context” at Caulfield’s blog, Hapgood, and as part of the OSU Libraries’ guide to media literacy. (Full disclosure: Mike is a former colleague from Washington State University Vancouver.)

If I have one complaint about Caulfield and Wineburg’s book, it’s that it dwells at length on the particulars of analyzing Google search results, which fill pages of accompanying figures and a whole chapter on the search engine as “the bestie you thought you knew” (p. 49). To be sure, Google still occupies a large share of the time students and faculty spend online. But as in my quest for learning norms protocols, readers are already turning to large language model tools for help in deciding what to believe online. In that respect, I find other chapters in Verified (on scholarly sources, the rise of Wikipedia, deceptive videos, and so-called native advertising) more useful. And if you go there, don’t miss the author’s final take on the power of emotion in finding the truth—a line that sounds counterintuitive, but in context adds another, rather moving dimension to the case against checklists.

Given the acceleration of machine learning, will lateral reading and SIFTing hold up in the age of AI? Caulfield and Wineburg certainly think so. Building out context becomes all the more necessary, they write in a postscript on the future of verification, “when the prose on the other side is crafted by a convincing machine” (p. 221). On that note, I invite you and your students to try out some of these moves on your favorite chatbot.

Another Postscript

The other day, I gave Microsoft’s AI-powered search engine a few versions of the same prompt I had put to ChatGPT. In “balanced” mode, Bing dutifully recommended resources from Stanford, Cornell, and Harvard on introducing norms for learning in online college classes. Over in “creative” mode, Bing’s synthesis was slightly more offbeat—including an early-pandemic blog post on setting norms for middle school faculty meetings in rural Vermont. More importantly, the bot wasn’t hallucinating. Most of the sources it suggested seemed worth investigating. Pausing before each rabbit hole, I took a deep breath.

Related Resource

Oregon State Ecampus recently rolled out its own AI toolkit for faculty, based on an emerging consensus that developing capacities for using this technology will be necessary in many areas of life. Of particular relevance to this post is a section on AI literacy, conceptualized as “a broad set of skills that is not confined to technical disciplines.” As with Verified, I find the toolkit’s frameworks and recommendations on teaching AI literacy particularly helpful. For instance, if students are allowed to use ChatGPT or Bing to brainstorm and evaluate possible topics for a writing assignment, “faculty might provide an effective example of how to ask an AI tool to help, ideally situating explanation in the context of what would be appropriate and ethical in that discipline or profession.”

References

Caulfield, M., & Wineburg, S. (2023). Verified: How to think straight, get duped less, and make better decisions about what to believe online. University of Chicago Press.

Mollick, E. (2023, July 15). How to use AI to do stuff: An opinionated guide. One Useful Thing.

Oregon State Ecampus. (2023). Artificial Intelligence Tools.

Media Literacy in the Age of AI, Part I: “You Will Need to Check It All”

Posted on by | Leave a reply

Have you found yourself worried or overwhelmed in thinking about the implications of artificial intelligence for your discipline? Whether, for example, your department’s approaches to teaching basic skills such as library research and source evaluation still hold up? You’re not alone. As we enter another school year, many educators continue to think deeply about questions of truth and misinformation, creativity, and how large language model (LLM) tools such as chatbots are reshaping higher education. Along with our students, faculty (oh, and instructional designers) must consider new paradigms for our collective media literacy.

Here’s a quick backstory for this two-part post. In late spring, shortly after the “stable release” of ChatGPT to iOS, I started chatting with bot model GPT-3.5, which innovator Ethan Mollick describes as “very fast and pretty solid at writing and coding tasks,” if a bit lacking in personality. Other, internet-connected models, such as Bing, have made headlines for their resourcefulness and darker, erratic tendencies. But so far, access to GPT-4 remains limited, and I wanted to better understand the more popular engine’s capabilities. At the time, I was preparing a workshop for a creative writing conference. So, I asked ChatGPT to write a short story in the modern style of George Saunders, based in part on historical events. The chatbot’s response, a brief burst of prose it titled “Language Unleashed,” read almost nothing like Saunders. Still, it got my participants talking about questions of authorship, originality, representation, etc. Check, check, check.

The next time I sat down with the GPT-3.5, things went a little more off-script.

One faculty developer working with Ecampus had asked our team about establishing learning norms in a 200-level course dealing with sensitive subject matter. As a writing instructor, I had bookmarked a few resources in this vein, including strategies from the University of Colorado Boulder. So, I asked ChatGPT to create a bibliographic citation of Creating Collaborative Classroom Norms, which it did with the usual lightning speed. Then I got curious about what else this AI model could do, as my colleagues Philip Chambers and Nadia Jaramillo Cherrez have been exploring. Could ChatGPT point me to some good resources for faculty on setting norms for learning in online college classes?

“Certainly!” came the cheery reply, along with a summary of five sources that would provide me with “valuable information and guidance” (see Image 1). Noting OpenAI’s fine-print caveat (“ChatGPT may produce inaccurate information about people, places, or facts”), I began opening each link, expecting to be teleported to university teaching centers across the country. Except none of the tabs would load properly.

“Sorry we can’t find what you’re looking for,” reported Inside Higher Ed. “Try these resources instead,” suggested Stanford’s Teaching Commons. A closer look with Internet Archive’s Wayback Machine confirmed that the five sources in question were, like “Language Unleashed,” entirely fictitious.

An early chat with ChatGPT-3.5, asking whether the chatbot can point the author to some good resources for faculty on setting classroom norms for learning in online college classes. "Certainly," replies ChatGPT, in recommending five sources that "should provide you with valuable information and guidance."

Image 1: An early, hallucinatory chat with ChatGPT-3.5

As Mollick would explain months later: “it is very easy for the AI to ‘hallucinate’ and generate plausible facts. It can generate entirely false content that is utterly convincing. Let me emphasize that: AI lies continuously and well. Every fact or piece of information it tells you may be incorrect. You will need to check it all.”

The fabrications and limitations of chatbots lacking real-time access to the ever-expanding web have by now been well-documented. But as an early adopter, the speed and confidence ChatGPT brought to the task of inventing and describing fake sources felt unnerving. And without better guideposts for verification, I expect students less familiar with the evolution of AI will continue to experience confusion, or worse. As the Post recently reported, chatbots can easily say offensive things and act in culturally-biased ways—”a reminder that they’ve ingested some of the ugliest material the internet has to offer, and they lack the independent judgment to filter that out.”

Just how, exactly, we’re supposed to “check it all” happens to be the subject of a lively, forthcoming collaboration from two education researchers who have been following the intersection of new media and misinformation for decades.

Stay tuned for an upcoming post with the second installment of “Media Literacy in the Age of AI,” a review of Verified: How to Think Straight, Get Duped Less, and Make Better Decisions about What to Believe Online by Mike Caulfield and Sam Wineburg (University of Chicago Press, November 2023).

References

Mollick, E. (2023, July 15). How to use AI to do stuff: An opinionated guide. One Useful Thing.

Wroe, T., & Volckens, J. (2022, January). Creating collaborative classroom norms. Office of Faculty Affairs, University of Colorado Boulder.

Yu Chen, S., Tenjarla, R., Oremus , W., & Harris, T. (2023, August 31). How to talk to an AI chatbot. The Washington Post.

Valuable Skills for Online Students and Online Instructors

Posted on by | Leave a reply

By Cat Turk and Mary Ellen Dello Stritto

In this time of rapid change in online education, we can benefit from leveraging the expertise of faculty who have experienced the evolution of online education. At the Oregon State University (OSU) Ecampus Research Unit, we have been learning from a group of instructors who have taught online for ten years or more. A review of recent research uncovered that these instructors are an untapped resource. Their insights can provide valuable guidance for instructors who are just beginning their careers or instructors who may be preparing to teach online for the first time. Further, their perspectives can also be enlightening for online students.

In 2018-2019 we conducted interviews with 33 OSU faculty who had been teaching online for 10 years or more as a part of a larger study. Two of the questions we asked them were the following:

  1. What skills do you think are most valuable for online instructors to have?
  2. What skills do you think are most valuable for online students to have?

We will share some of the results of a qualitative analysis of these questions and highlight the similarities and differences.

When asked about the most valuable skills for online instructors, three key skills emerged: communication, organization, and time management. When asked about the most valuable skills for online students to have, the same skills were among the most frequently mentioned by these instructors.

As the table below shows, in the responses about skills for online instructors, communication emerged as the most prominent skill, with 85% of instructors in the study emphasizing its importance, while time management and organization were split evenly at 45%. In their response about skills for students, 64% of the instructors emphasized both communication and time management, while 42% discussed organization. When discussing communication for instructors, they indicated that effective communication is essential for building rapport with students, providing clear instructions, and facilitating meaningful interactions in the online environment. Organization (such as structuring course materials or their weekly work process) and time management skills (such as scheduling availability to connect with students) were also highly valued by these instructors. Read more about the analysis of instructor skills here.

 Skills for InstructorsSkills for Students
Communication    28 responses (85%)   21 responses (64%)
Time Management15 responses (45%)  21 responses (64%)
Organization15 responses (45%)   14 responses (42%)
Self-Motivation   —21 responses (64%)            
Frequency of responses of skills for instructors and students.

The responses to both questions emphasized the significance of communication skills in written assignments and in proactive connections within the scope of the online learning environment. Instructors articulated that online students needed to be proactive communicators themselves. Examples of this include contacting their instructors about questions and clarification in a timely way, interacting with their peers in a respectful manner, and turning in quality written assignments that demonstrate comprehension of their learning material. For students, clear and effective communication ensures understanding and engagement, while organization facilitates seamless navigation through course materials, and time management ensures that students are able to make the most of the asynchronous environment.

While time management and organization were both considered by instructors to be just as crucial for students, their responses demonstrated that these skills were needed for different reasons than would be the case for instructors. Instructors personally valued time management and organization due to the nature of facilitating courses online. When the online classroom can travel from place to place, setting blocks of intentional time and structuring hours accordingly were considered essential to instructors for maintaining a work-life balance and so tasks would not be missed.

On the other hand, according to these instructors, students need time management and organization due to the asynchronous and sometimes isolating nature of online courses. One instructor stressed that:

 “[Students] do need to be more organized than on-ground students, because there’s not that weekly meeting to keep students on track.”

These instructors indicated some online students may need to structure their study time to accommodate a different time zone, while others may need to structure their academic pursuits around careers or children. Another instructor emphasized that:

“A lot of our [online students] actually work full-time, so they have families and kids and have to be much more organized too.”

While there were overlaps with the responses to the two questions, a notable difference was the emergence of another skill for students: self-motivation. This concept of self-motivation emerged from the instructor responses about students’ capacity to persevere in online courses. This included their level of motivation, capacity to learn on their own, and comfort with self-paced learning.

One instructor said the following about students’ self-motivation,

“Some people would say it’s self-discipline, but I think it’s more of they have to have a purpose for that class.”

Self-motivation was not mentioned by the instructors as a skill for online instructors, suggesting that these instructors perceive this as more pertinent to students for success in managing their own learning process. It is worth noting that proactive communication was highlighted as an essential aspect of self-motivation, with instructors emphasizing that students who take the initiative in reaching out to them tend to be more successful. This observation suggests that self-motivated individuals are more likely to actively seek support and clarification, which can enhance their learning experience and overall success. 

Another noteworthy aspect was the need for students to be comfortable with learning in physical isolation. Instructors acknowledged that online learners must navigate the challenges of studying independently without the immediate presence of peers and instructors. For online students specifically,

“They need to be motivated because they’re not going to have peers sitting in a classroom with them, and they don’t have a place that they have to physically go every week.”

This finding underscores the importance of maintaining motivation and engagement, as students ideally possess an intrinsic drive to succeed despite the absence of a physical connection to the university and their classmates.

The findings from this study highlight the importance of certain similar skills for both online instructors and students. Effective communication, organization, and time management are vital for success in the online learning environment for both instructors and students. We found this to be an interesting connection that online students might benefit from understanding: these are key skills that students and instructors have in common.

Our findings about self-motivation may be useful for online instructors. Consider incorporating strategies that foster student self-motivation, such as goal-setting exercises, regular check-ins, and providing opportunities for self-reflection. By empowering students to take ownership of their learning, instructors might enhance student engagement and success in the online environment.

Further, students can learn from the instructors’ emphasis on communication, organization, and time management skills. They can intentionally work on improving their communication skills, seeking clarification when needed, and actively participating in online discussions. Developing effective organization and time management strategies, such as creating schedules, prioritizing tasks, and breaking them down into manageable chunks, may significantly enhance their online learning experience.

The field of online education is evolving rapidly, and here we can see how educators and students alike are adapting to these changes. The experiences of long-term online instructors provide valuable insights into the skills necessary for success in the online learning environment. In the future, what answers would we find if we asked students the same question: what do online students think are the skills needed to succeed in the online classroom? By understanding the shared and distinct perspectives of instructors and students, educators can design effective online courses and support systems that foster meaningful learning experiences and empower students to succeed.

Office Hours: Welcoming our online students to the virtual table

Posted on by | Leave a reply
Image of a black chair in an empty room

Have you implemented office hours in your online course, with few students taking advantage of that time to connect? This can often seem like a mystery, when we hear so often from Ecampus students that they desire to build deeper relationships with their instructors. Let’s dive into some of the reasons why online students may be hesitant to attend and identify a few ways we can improve this in our courses. 

Photo by Paula Schmidt via Pexels

Who are our learners?

To help us address this question, let’s first consider who our learners are. The vast majority of Ecampus students are working adults who complete coursework in the evenings and on weekends, outside of regular business hours.

Ecampus learners reside in all 50 states and more than 60 countries. The people who enroll in Ecampus courses and programs consist of distance (off-campus) students — whose life situations make it difficult for them to attend courses on Oregon State’s Corvallis or Bend campuses — and campus-based students who may take an occasional online course due to a schedule conflict or preference for online learning. Here is a student demographic breakdown for the 2021-22 academic year: Approximately 26% of OSU distance students live in Oregon. The average age of OSU distance students is 31.

Data shared from Ecampus News

Considering this data about our online population, along with qualitative survey data and insights from our student success team, we can also deduce some additional factors. Our students are:

  • Working professionals, balancing family and personal commitments
  • Concerned about time
  • Often feel stressed and overwhelmed
  • Seeking flexibility and understanding
  • Located in a variety of time zones, with mixed schedules
  • From a number of cultural backgrounds and perspectives
  • Looking to identify the value of tasks/assignments and seeking how their education will benefit them personally
  • May experience self-doubt, imposter syndrome, or hesitancy around their ability to successfully complete their program

Identifying the barriers

Now that we have a better understanding of our online learners and some of the challenges they face, let’s consider how they approach office hours. The slide below, shared at a TOPS faculty workshop in 2020, outlines some of the self-reported reasons that students may not be engaging in support sessions or reaching out for help.

Slides shared from TOPS presentation, February 2020, by Brittni Racek

Students may have the connotation that office hours are for ‘certain types of concerns’ and not see it as a time to connect with their instructor on other areas of interest (i.e. graduate school, career planning, letters of recommendation, etc.) They may also see it as a sign of weakness or fault, rather than a strength for being able to utilize that time to build a relationship or increase their learning. Students may have also had past experiences, at OSU or elsewhere, that have formed an understanding of what office hours entail and what is allowed at these meetings.

Student feedback

In the Ecampus Annual Survey (2020), when asked about faculty behavior that made them feel comfortable attending office hours, students shared that instructor friendliness, promptly answering student questions, providing accessible and flexible office hour options, and demonstrating strong communication throughout the course were specifically helpful in encouraging use of office hours. (Ecampus Annual Student Survey)

Those who had not taken advantage of office hours shared reasons that generally fell into four categories:

  1. Office hours conflicted with life and were not accessible to them
  2. The student had not yet needed to use office hours
  3. Using other forms of communication to ask for help
  4. Lack of awareness of if or when office hours were offered

Alternative approaches

Rename and reframe ‘office hours’

To help students identify the purpose of your Office Hours time, and to make it a little less intimidating, you might consider renaming these hours. Some ideas include Student Hours, Homework Help, Ask Me Anything Hours, Virtual Coffee Chat, etc. Some instructors separate times for course related questions from times that are more for connection and talking about outside topics such as current industry news, future planning, etc.

It’s important to be clear with students what they can discuss with you at these times, and to also encourage their participation and welcome it. You could do this by choosing intentional wording in the way you share your hours, and also sending reminders by announcement or direct message.

Offer flexibility

To help make your hours accessible to a variety of students, you might consider offering a number of different times throughout the term, staggering when those are available (i.e. morning, lunch, evening, or a weekend day). You can also offer the option to request office hours by appointment.

Another strategy would be to survey your students at the beginning of the term to see when the best times are for the majority of the class. You could also leverage this survey to ask about topics of interest or to see if they have any concerns or questions starting off the term.

Consider the tools

For synchronous virtual meetings, we would recommend using Zoom as most OSU students are comfortable with this tool, and everyone has free access to it. Zoom links can be shared, and also integrated into your Canvas course using the tool in the Canvas menu. For asynchronous questions, you might create a Q&A forum for each week or module of the course (and subscribe to ensure timely notification). If you are using Canvas messaging, we recommend outlining that in your communication plan so that students know the best way to reach you.

Some instructors have also experimented with outside tools, such as Gather. Gather is a platform for building digital spaces for teams to connect at a distance. It is free to use for spaces that allow up to 25 users at once. You can chat, enable your mic and camera for audio/video interactions, and create specific areas for small group conversations.

Demonstrate care and community

One of the best strategies for encouraging students to utilize your meeting hours or to reach out for help in other ways, is to demonstrate care throughout your course. This can be done by using welcome and inclusive language in your Syllabus and written course content, having a warm and friendly tone in your media (i.e. recorded lectures and videos), and reaching out proactively to students who may be low in participation or struggling academically.

Resources

  • Office Hours for Online Courses – This guide was created by our Ecampus Faculty Support team, and provides a great overview for best practices and implementation.
  • Office Hours Explainer – This PDF was designed by OSU’s Academic Success Center, as a student-facing resource on Office Hours. It explains the variety of topics available, steps to take, and preparation for the student. There is a specific section about online courses, but the majority of the guide is applicable to Ecampus students.
  • Effective Office Hours – This faculty guide, created by the Center for Academic Innovation at the University of Michigan, offers some ideas for how to leverage virtual office hours, including specific strategies from an instructional perspective.

Learning Outcomes: Do Students Even Read Them?

Posted on by | Leave a reply

Learning outcomes (LOs) are used in instructional design to describe the skills and knowledge that students should have at the end of a course or learning unit, and to design assessments and activities that support these goals. It is widely agreed that specific, measurable outcomes are essential for planning instruction; however, some educators question the benefits of explicitly presenting them to students. I have been asked (and wondered myself): “What is the point of listing learning outcomes in the course?” “How do they help learning? “Do students even read them?”

So, I went on a quest for research that attempted to answer such questions. I was particularly interested in unit/module-level outcomes, as those are the ones that directly steer the content, and students see them throughout the course. Here’s a brief summary of what I found.

Note: the studies use the terms “learning outcome”, “learning objective”, or “learning goal” – they all refer to the same concept: a specific and measurable description of the skills and knowledge that students are expected to have at the end of a learning unit/period of study. At OSU we use the term “outcomes”.

What Does the Research Say?

Armbruster et al. (2009) redesigned an Introductory Biology course at Georgetown University, Washington, DC, using active learning and student-centered pedagogies, leading to increased student performance and satisfaction. One of the strategies used was to include explicit learning goals in the lecture slides, and labeling exam and quiz questions with the related goals. Students’ attitudes towards the course were assessed via a questionnaire and comparison of university-administered student evaluations. Students were asked to rank lecture components in terms of helpfulness to learning, and the authors found that one of the highest-ranking elements was the inclusion of explicit learning goals.

Simon and Taylor (2009) surveyed 597 students from computer science and microbiology and immunology courses at the University of British Columbia, where instructors presented learning goals at the beginning of each lecture or topic area. The questions were open and the answers coded into a number of categories, which helped them identify several values of goals. The main value was “knowing what I need to know”: students reported that the goals showed them how to focus their efforts and felt that the goals “allowed them to organize the information more effectively and be more expertlike in their approach to the class” (Simon & Taylor, 2009, p.55). The authors did not find any difference between presenting the goals before each lecture versus at the beginning of the unit/topic area.

Brooks et al. (2014) examined students’ views of learning outcomes at the University of Leicester, UK. First, they surveyed 918 students taking Biological Sciences, English and Medicine courses. They found that 81% of participants agreed or strongly agreed that learning outcomes are useful learning aids. Additionally, 46% found LOs more useful as their courses progressed, and 49% reported that they engaged more with the LOs as the course progressed. The authors also investigated when LOs are most useful, and found that the most common answer (46%) was when reviewing the material. Moreover, 49% of students reported that LOs can only be fully understood at the end of a module. The researchers followed up on these results with a focus group, which confirmed that students use LOs in various ways and at various points during the course.

Osueke et al. (2018) looked into students’ use and perceptions of learning objectives at University of Georgia. 185 students in an undergraduate Introduction to Biochemistry and Molecular Biology course took part in the study. The instructors included instructions in the syllabus, which they also stated on the first day of class: “Focus on the learning objectives. The exams will assess your accomplishment of the learning objectives. Use the learning objectives as a guide for what to focus on when you are completing assignments and studying for exams.” Students completed two assignments requiring them to explain their use of the LOs. The researchers found that many students (33.8%) reported they had been instructed on how to use LOs to study – these instructions ranged from passively “look over” to using them as a study guide. The ways students used the LOs were: as questions to answer (47.4%), as a resource for studying (24.1%), as a self-assessment tool (14.3%), and passive use (13.5%). When asked why they find the LOs helpful, students said that they help them: narrow down the information (57.1%); organize their studying (23.3%); communicate information (5.3%); monitor their understanding (4.5%); forced them to study (1.5%).

Sana et al. (2020) conducted three experiments aiming to find to what extent presenting the LOs improve retention of information. Participants were asked to read five passages on a neuroscience topic, and then they were tested on comprehension and retention. The experiments took place at McMaster University, Ontario and employed different participants, methods, materials, and procedures. They found that: interpolating LOs throughout the lesson (as opposed to all LOs presented at the beginning) improved learning compared to not including LOs, especially when students’ attention was explicitly directed to them; converting LOs into pretest questions (that students attempted to answer) further enhanced performance; multiple-choice and short answer questions were equally effective; and withholding feedback on pretests was more effective than providing feedback – the explanation proposed by the authors for this last finding was that students may be more motivated to seek the correct answers themselves, which causes further processing of the material.

Barnard et al. (2021) investigated students’ and academics’ perspectives on the purpose of learning objectives and approaches to assessment preparation. They conducted focus groups with participants from an undergraduate Psychology course at the University of Nottingham, UK. The students reported that LOs are useful for guidance, as they “use them to create direction for some of the learning and revision strategies” (Barnard et al., 2021, p. 679).

Conclusions and Recommendations

Good news! The findings of these studies suggest that many students do appreciate clear LOs and use them to guide their learning. The LOs help them understand what they are expected to know – thus, students use them to focus their study, to review for an exam, and to self-check their knowledge.

As instructors and instructional designers, what can we do to help students take full advantage of LOs? Apart from having specific and measurable LOs, make sure that the LOs are well aligned with the activities, and make this alignment explicit. It may also be helpful to offer some guidance on how to use the LOs, for instance by prompting students to recap their learning at the end of a unit based on the LOs. Finally, we could turn the LOs into questions and use them as a pretest.

For more on creating and using LOs, check out the CBE—Life Sciences Education website, which has an informative guide, including a section on student use. 

Do you have any other ideas or resources on how to use learning outcomes to improve students’ experience and study habits? If so, we’d love to hear from you!

References

Armbruster, P., Patel, M., Johnson, E., & Weiss, M. (2009). Active learning and student-centered pedagogy improve student attitudes and performance in Introductory Biology. CBE Life Sciences Education, 8(3), 203–213. https://doi.org/10.1187/cbe.09-03-0025

Barnard, M., Whitt, E., & McDonald, S. (2021). Learning objectives and their effects on learning and assessment preparation: Insights from an undergraduate psychology course. Assessment and Evaluation in Higher Education, 46(5), 673–684. https://doi.org/10.1080/02602938.2020.1822281

Brooks, S., Dobbins, K., Scott, J. J. A., Rawlinson, M., & Norman, R. I. (2014). Learning about learning outcomes: The student perspective. Teaching in Higher Education, 19(6), 721–733. https://doi.org/10.1080/13562517.2014.901964

Osueke, B., Mekonnen, B., & Stanton, J. D. (2018). How undergraduate science students use learning objectives to study. Journal of Microbiology & Biology Education, 19(2). https://doi.org/10.1128/jmbe.v19i2.1510

Sana, F., Forrin, N. D., Sharma, M., Dubljevic, T., Ho, P., Jalil, E., & Kim, J. A. (2020). Optimizing the efficacy of learning objectives through pretests. CBE Life Sciences Education, 19(3), ar43–ar43. https://doi.org/10.1187/cbe.19-11-0257

Simon, B., & Taylor, J. (2009). What is the value of course-specific learning goals? Journal of College Science Teaching, 39(2), 52–57. Retrieved from: https://www.colorado.edu/sei/sites/default/files/attached-files/what_is_the_value_of_course-specific_learning_goals.pdf

Demonstrating Care: Reach Out & Refer

Posted on by | Leave a reply

As a new term begins, we are often thinking about the logistics of our courses, the Syllabus and course schedule, and ensuring everything is working properly. For our students, these early weeks set the tone for what they might expect from their courses and from their instructors. Your first announcement, the language and tone in the Syllabus, how you greet incoming students – these small actions all help to create a welcoming environment for your course. When students feel included in a positive course climate, they are more motivated and engaged in learning.

In the weeks ahead, some students will likely reach out to you with concerns or information about major events going on in their lives. Faculty are often the first to hear of health issues, death in the family, deployment, financial matters, and a variety of mental health concerns and needs. In prior surveys, Ecampus students have shared that the most important relationship in their college career is with their instructor(s), rated higher than their advisors or other student support professionals around campus. When life happens, you are often the first person a student thinks to reach out to for support and direction. Last year, Ecampus put forth the Online Teaching Principles, derived from research-based best practices. The principle “Reach Out and Refer” directly relates to what we can do when our students need some additional support.

Check in with students who may be struggling, and refer students to the appropriate technology, academic or student support services in response to their articulated or observed needs.

Oregon State University Ecampus, Online Teaching Principle: Reach Out And Refer

When students reach out, your care, concern for their well-being, and support is sometimes enough to help the student. That may look like an assignment extension, acknowledgement of their circumstances, setting up a time to speak, or a variety of other measures. At other times, there are situations when making a referral to the appropriate resource or department is the best course of action. In these instances, it is important to remain calm and formulate a plan.

OSU’s Counseling & Psychological Services (CAPS) department shares the following about making student referrals:

When to Make A Referral

There are situations when making a referral is the best option for both you and the student. For example:

  • You know that you can’t handle the request or the behavior. There are limits to the kinds of help a faculty or staff member can provide.
  • You believe that personality differences will interfere with your ability to help.
  • You know the student personally and believe that you could not be objective.
  • You feel overwhelmed, pressed for time, or stressed.
  • The student acknowledges a problem but is reluctant to discuss it with you.
  • After working with the student for some time, you realize that you don’t know how to proceed.
  • The student’s problems are better handled through services such as CAPS, Financial Aid, the Registrar’s Office, Affirmative Action, or Legal Advising.

How to Make a Referral

Some people accept a referral for professional help more easily than others do.  Here are some tips for making a successful referral.

  • Let the student know that it is not necessary to know exactly what is wrong in order to seek assistance.
  • Assure the student that seeking help does not necessarily mean that their problems are unusual or extremely serious.
  • Be frank with students about your own limits of time, energy, training, objectivity, and willingness to help.
  • If appropriate, suggest that the student consider talking with family members, friends, clergy, community agencies, and campus offices.

CAPS provides consultations to faculty and staff who have urgent concerns about a student. If you have an immediate need, please call 541-737-2131.  Phone counselors are available after hours. If you or a person of concern are experiencing an emergency, please call 911 off campus or 541-737-7000 on campus.

The Student Care Team has compiled a chart (pictured below) of Resources For Consultation and Referral for AY 22 that can be referenced via their Box folder.

Resources for instructors

There are a wide variety of concerns that a student may bring to you. It can be time-consuming to identify the available resources and get students to the right area. There are a few main webpages you can bookmark that outline the resources available to our Ecampus students.

  1. Student Resources For Ecampus Students – This page on the Ecampus website maintains a comprehensive list of all resources available to Ecampus students. It includes academic resources, emergency food and housing, disability access services, mental health, technical support, and more. This is a great page to bookmark and/or print the PDF version that is linked at the bottom of the webpage.
  2. Student Care Team – This Box folder contains resources for faculty including a referral and consultation chart and tips for working with distressed students.
  3. In Crisis Support For Students (CAPS) – 24/7 support for students in crisis. Includes contact information for CAPS, Suicide & Crisis Lifeline, and more.
  4. If You Are Concerned About A Student (CAPS) – Faculty/staff member consultation form. You can also call 541-737-2131 for a more immediate response.
  5. Ecampus Student Services – If your student is not in crisis, but you are unsure where to start, directing them to our student services representatives is a great option. They assist students with navigating OSU resources and are the first point of contact for student inquiries. Phone: 800-667-1465 (select option 1) or Email: ecampus.ess@oregonstate.edu.
  6. Ecampus Student Success Coaching – If you feel that your student(s) could benefit from individualized, strengths-based academic counseling, you can refer them to the success coaching team. This group works with all undergraduate Ecampus students.