Learning outcomes (LOs) are used in instructional design to describe the skills and knowledge that students should have at the end of a course or learning unit, and to design assessments and activities that support these goals. It is widely agreed that specific, measurable outcomes are essential for planning instruction; however, some educators question the benefits of explicitly presenting them to students. I have been asked (and wondered myself): “What is the point of listing learning outcomes in the course?” “How do they help learning? “Do students even read them?”

So, I went on a quest for research that attempted to answer such questions. I was particularly interested in unit/module-level outcomes, as those are the ones that directly steer the content, and students see them throughout the course. Here’s a brief summary of what I found.

Note: the studies use the terms “learning outcome”, “learning objective”, or “learning goal” – they all refer to the same concept: a specific and measurable description of the skills and knowledge that students are expected to have at the end of a learning unit/period of study. At OSU we use the term “outcomes”.

What Does the Research Say?

Armbruster et al. (2009) redesigned an Introductory Biology course at Georgetown University, Washington, DC, using active learning and student-centered pedagogies, leading to increased student performance and satisfaction. One of the strategies used was to include explicit learning goals in the lecture slides, and labeling exam and quiz questions with the related goals. Students’ attitudes towards the course were assessed via a questionnaire and comparison of university-administered student evaluations. Students were asked to rank lecture components in terms of helpfulness to learning, and the authors found that one of the highest-ranking elements was the inclusion of explicit learning goals.

Simon and Taylor (2009) surveyed 597 students from computer science and microbiology and immunology courses at the University of British Columbia, where instructors presented learning goals at the beginning of each lecture or topic area. The questions were open and the answers coded into a number of categories, which helped them identify several values of goals. The main value was “knowing what I need to know”: students reported that the goals showed them how to focus their efforts and felt that the goals “allowed them to organize the information more effectively and be more expertlike in their approach to the class” (Simon & Taylor, 2009, p.55). The authors did not find any difference between presenting the goals before each lecture versus at the beginning of the unit/topic area.

Brooks et al. (2014) examined students’ views of learning outcomes at the University of Leicester, UK. First, they surveyed 918 students taking Biological Sciences, English and Medicine courses. They found that 81% of participants agreed or strongly agreed that learning outcomes are useful learning aids. Additionally, 46% found LOs more useful as their courses progressed, and 49% reported that they engaged more with the LOs as the course progressed. The authors also investigated when LOs are most useful, and found that the most common answer (46%) was when reviewing the material. Moreover, 49% of students reported that LOs can only be fully understood at the end of a module. The researchers followed up on these results with a focus group, which confirmed that students use LOs in various ways and at various points during the course.

Osueke et al. (2018) looked into students’ use and perceptions of learning objectives at University of Georgia. 185 students in an undergraduate Introduction to Biochemistry and Molecular Biology course took part in the study. The instructors included instructions in the syllabus, which they also stated on the first day of class: “Focus on the learning objectives. The exams will assess your accomplishment of the learning objectives. Use the learning objectives as a guide for what to focus on when you are completing assignments and studying for exams.” Students completed two assignments requiring them to explain their use of the LOs. The researchers found that many students (33.8%) reported they had been instructed on how to use LOs to study – these instructions ranged from passively “look over” to using them as a study guide. The ways students used the LOs were: as questions to answer (47.4%), as a resource for studying (24.1%), as a self-assessment tool (14.3%), and passive use (13.5%). When asked why they find the LOs helpful, students said that they help them: narrow down the information (57.1%); organize their studying (23.3%); communicate information (5.3%); monitor their understanding (4.5%); forced them to study (1.5%).

Sana et al. (2020) conducted three experiments aiming to find to what extent presenting the LOs improve retention of information. Participants were asked to read five passages on a neuroscience topic, and then they were tested on comprehension and retention. The experiments took place at McMaster University, Ontario and employed different participants, methods, materials, and procedures. They found that: interpolating LOs throughout the lesson (as opposed to all LOs presented at the beginning) improved learning compared to not including LOs, especially when students’ attention was explicitly directed to them; converting LOs into pretest questions (that students attempted to answer) further enhanced performance; multiple-choice and short answer questions were equally effective; and withholding feedback on pretests was more effective than providing feedback – the explanation proposed by the authors for this last finding was that students may be more motivated to seek the correct answers themselves, which causes further processing of the material.

Barnard et al. (2021) investigated students’ and academics’ perspectives on the purpose of learning objectives and approaches to assessment preparation. They conducted focus groups with participants from an undergraduate Psychology course at the University of Nottingham, UK. The students reported that LOs are useful for guidance, as they “use them to create direction for some of the learning and revision strategies” (Barnard et al., 2021, p. 679).

Conclusions and Recommendations

Good news! The findings of these studies suggest that many students do appreciate clear LOs and use them to guide their learning. The LOs help them understand what they are expected to know – thus, students use them to focus their study, to review for an exam, and to self-check their knowledge.

As instructors and instructional designers, what can we do to help students take full advantage of LOs? Apart from having specific and measurable LOs, make sure that the LOs are well aligned with the activities, and make this alignment explicit. It may also be helpful to offer some guidance on how to use the LOs, for instance by prompting students to recap their learning at the end of a unit based on the LOs. Finally, we could turn the LOs into questions and use them as a pretest.

For more on creating and using LOs, check out the CBE—Life Sciences Education website, which has an informative guide, including a section on student use. 

Do you have any other ideas or resources on how to use learning outcomes to improve students’ experience and study habits? If so, we’d love to hear from you!

References

Armbruster, P., Patel, M., Johnson, E., & Weiss, M. (2009). Active learning and student-centered pedagogy improve student attitudes and performance in Introductory Biology. CBE Life Sciences Education, 8(3), 203–213. https://doi.org/10.1187/cbe.09-03-0025

Barnard, M., Whitt, E., & McDonald, S. (2021). Learning objectives and their effects on learning and assessment preparation: Insights from an undergraduate psychology course. Assessment and Evaluation in Higher Education, 46(5), 673–684. https://doi.org/10.1080/02602938.2020.1822281

Brooks, S., Dobbins, K., Scott, J. J. A., Rawlinson, M., & Norman, R. I. (2014). Learning about learning outcomes: The student perspective. Teaching in Higher Education, 19(6), 721–733. https://doi.org/10.1080/13562517.2014.901964

Osueke, B., Mekonnen, B., & Stanton, J. D. (2018). How undergraduate science students use learning objectives to study. Journal of Microbiology & Biology Education, 19(2). https://doi.org/10.1128/jmbe.v19i2.1510

Sana, F., Forrin, N. D., Sharma, M., Dubljevic, T., Ho, P., Jalil, E., & Kim, J. A. (2020). Optimizing the efficacy of learning objectives through pretests. CBE Life Sciences Education, 19(3), ar43–ar43. https://doi.org/10.1187/cbe.19-11-0257

Simon, B., & Taylor, J. (2009). What is the value of course-specific learning goals? Journal of College Science Teaching, 39(2), 52–57. Retrieved from: https://www.colorado.edu/sei/sites/default/files/attached-files/what_is_the_value_of_course-specific_learning_goals.pdf

Four students working together on a project
Four students working together on a project

A term paper is a common final assignment, but does the final assignment have to be a paper? The answer depends on the type of course and the learning outcomes. If the final assignment can be an alternative to the term paper, we can consider other types of assignments that allow students not only to accomplish the learning outcomes, as expected, but also to engage more deeply with the content and exercise critical thinking. A caveat related to the discipline is important here. Fields that require a writing component may necessarily rely on the term paper which can be scaffolded through a set of stages. For assignments with a sequence of tasks, refer to staged assignments (Loftin, 2018) for details on how to design them.

A first step in moving towards considering other types of assessments is to self-reflect on the purpose of the course and what role it will play in students’ learning journeys. You can use some of the following questions as a guide to self-reflect:

Focus levelInitial self-questions
Course What is the nature of the course (e.g., practice-based, reading-intense, general education, writing-intensive, labs, etc.)?
What are the outcomes?
What level do the outcomes target (e.g., recall, analysis, evaluation)?
Discipline What do people in the discipline I teach regularly do in the work environment? Do they: write grants? or develop lesson plans? write technical reports? write articles or white papers? build portfolios? demonstrate skills? and so on…
Do all students need to complete the final assignment in the same format or can the format vary (e.g., paper, presentation, podcast)?

Taking some time to reevaluate the assessment practices in your course might be beneficial for your students who seek meaningful learning opportunities and expect relevant assignments (Jones, 2012; ). Students might also welcome variety and flexibility in how they learn and be evaluated (ASU Prep Digital, 2021; Soffer et al., 2019). 

Let’s explore alternative and authentic assessments next.

Alternative assessments

Alternative authentic assessments tend to focus on high-order and critical thinking skills –skills much appreciated these days. These assessments aim to provide more effective methods of increasing knowledge, fostering learning, and analyzing learning (Anderson, 2016; Gehr, 2021). Research also suggests that authentic assessments can increase students’ employability skills (Sotiriadou et al., 2019). However, the implementation of alternative assessments needs to transcend the status quo and become a critical element that allows instructors and students to focus on societal issues, acknowledge the value of assessment tasks, and embrace these assessments as vehicles for transforming society (McArthur, 2022). A student-centered environment also challenges educators to search for alternative assessments to make the learning experience more meaningful and lasting –fostering student agency and lifelong learning (Sambell & Brown, 2021).

Authentic assessments

I recall that when I was learning English, some of the types of practices and assessments did not really equip me to use the language outside the classroom. I thought that I would not go around the world and select choices from my interlocutors as I used to do through the language quizzes in class. I have been motivated by the Task-Based Language Teaching framework to focus on designing tasks (for learning and assessment) that help students use their knowledge and skills beyond the classroom –more useful and realistic tasks. 

Authentic assessments provide students with opportunities to apply what they learn to situations that they likely will encounter in their daily life. These situations will not be well-structured, easy to solve, and formulaic (like the English language practices I had); to the contrary, these situations will be complex, involve a real audience, require judgment, and require students to use a repertoire of skills to solve the problems and tasks (Wiley, n.d.). 

As you may see, alternative and authentic assessments can overlap, giving educators options to innovate their teaching and providing students opportunities to increase interest and engagement with their learning process. Below, you will see a collection of ideas for assessments that go beyond the term paper and give room for course innovations, learning exploration, and student agency.

Examples of Alternative and Authentic Assessments

You can select one or more assessments and create a sequence of assignments that build the foundation, give students an opportunity to reflect, and engage students in the active application of concepts. Diversifying the types of assessment practices can also serve as an inclusive teaching approach for your students to engage with the course in multiple ways (McVitty, 2022).

Introduction to New Concepts

students to these new ideas by designing simple and direct tasks such as:

  • Listen to podcasts, watch documentaries/films: write summaries or reviews
  • Conduct field observations: report what was observed, thoughts, and feelings
  • Create fact sheets and posters: share them with peers and provide comments
  • Study a case: write a report, design a visual abstract, create a data visualization or presentation
  • Create an infographic or digital prototype: present it to peers for feedback
  • Write a short newspaper article: contribute to the class blog, post it on the class digital board
  • Provide insights and comments: contribute with annotations and posts (e.g., Perusall, VoiceThread)

Reflective Practice

Reflection allows students to think further about their own learning process. If you are looking for activities to instill in students higher-order thinking skills and metacognitive skills, you can consider designing one of the tasks below. Remember to provide students with guiding questions for the reflection process

  • Review assignments and describe the learning journey: Create a portfolio with reflective notes
  • Develop an understanding of concepts by identifying areas of difficulty and feedforward goals: write a weekly learning log, create a learning journey map/graph 
  • Describe your learning experience through personal reflection: write an autoethnography
  • Connect course concepts and activities to learning experiences: create a think-out-loud presentation, podcast, or paper
  • Self-assess learning and progress: take a quiz, write a journal, create a learning map: “from here to there”)   

Theory Application

  • Demonstrate a solid understanding of key elements, theory strengths, and weaknesses: write an application paper to explore lines of inquiry, create an infographic connecting theory and examples, write an article or artifact critique through the lens of the theory
  • Dissect a theory by identifying and organizing the key components of theoretical frameworks: develop a theory profile document or presentation (instructor can create a dissect theory template)
  • Anchor course concepts in the literature: write a position paper, a response paper, or a commentary for a journal. 

Application Tasks

  • Guided interviews with professionals
  • Digital and augmented reality assets
  • Grant/funding applications
  • Project/conference proposals
  • Annotated bibliographies, article critiques
  • Reviews (e.g., music, videos, films, books, articles, media)
  • Oral discussion group exam (e.g., cases, scenarios, problem-solving) w/reflection
  • Conduct Failure Mode and Effect Analysis studies/simulations
  • Book newsletter, blog, and book live event Q&A (e.g., students plan the Q&A)
  • Create a student-led OER
  • Patchwork Screencast Assessment (PASTA) Reflections

The list of alternative and authentic assessments provided above is not exhaustive and I would welcome your comments and suggestions for the activities that you might have designed or researched for your online or hybrid courses. I would love to hear more about your approaches and thoughts on alternative and authentic assessments.

References

Anderson, M. (2016). Learning to choose, choosing to learn: The key to student motivation and achievement. ASCD.

ASU Prep Digital. (2021). Why Do Students Prefer Online Learning? https://www.asuprepdigital.org/why-do-students-prefer-online-learning/ 

Gher, L. (2021, March 11). How using authentic digital assessments can benefit students. Edutopia. https://www.edutopia.org/article/how-using-authentic-digital-assessments-can-benefit-students/#:~:text=With%20this%20method%20of%20assessment,of%20the%20comments%20and%20responses.

Jones, S. J. (2012). Reading between the lines of online course evaluations: Identifiable actions that improve student perceptions of teaching effectiveness and course value. Journal of Asynchronous Learning Networks, 16(1), 49-58. http://dx.doi.org/10.24059/olj.v16i1.227

Jopp, R., & Cohen, J. (2022). Choose your own assessment–assessment choice for students in online higher education. Teaching in Higher Education, 27(6), 738-755. https://doi.org/10.1080/13562517.2020.1742680

Loftin, D. (2018, April 24). Staged assignments. [Oregon State University Ecampus blog post] https://blogs.oregonstate.edu/inspire/2018/04/24/staged-assignments/

McArthur, J. (2022). Rethinking authentic assessment: work, well-being, and society. Higher Education, 1-17. https://link.springer.com/article/10.1007/s10734-022-00822-y

McVitty, D. (2022). Building back learning and teaching means changing assessment. Wonkhe Ltd.  

Soffer, T., Kahan, T. & Nachmias, R. (2019). Patterns of Students’ Utilization of Flexibility in Online Academic Courses and Their Relation to Course Achievement. International Review of Research in Open and Distributed Learning, 20(3). https://doi.org/10.19173/irrodl.v20i4.3949

Sotiriadou, P., Logan, D., Daly, A., & Guest, R. (2020). The role of authentic assessment to preserve academic integrity and promote skill development and employability. Studies in Higher Education, 45(11), 2132-2148. https://doi.org/10.1080/03075079.2019.1582015

Sambel, S., & Brown (2021). Covid-19 assessment collection. Assessment, Learning, and Teaching in Higher Education. https://sally-brown.net/kay-sambell-and-sally-brown-covid-19-assessment-collection/

Wiley University Services. (n.d.). Authentic assessment in the online classroom. https://ctl.wiley.com/authentic-assessment-in-the-online-classroom/

The following is a guest blog post from Meilianty Gunawan. Meili completed an Instructional Design internship with OSU Ecampus during Fall 2021.

Have you ever driven a car on a highway with no streetlights in the middle of the night? Your first instinct is to turn your car’s high-beam lights on to give you greater visibility on what lies far ahead of you. You are probably fine just by relying on the car’s low-beam lights, but you will not be able to get a clearer picture of the far end of the road as you would get from the high-beam.

Bloom’s Taxonomy is a framework used by instructors to develop course learning outcomes. It lays out six cognitive domains (from basic to more advanced) and examples of measurable action verbs along with those domains.

When course learning outcomes are set from the point of students’ knowledge and skills deficiency, students are inadvertently deprived of the greater things that they were able to achieve after completing the course. Lower-division courses (100- and 200-level courses) generally focus on the lower-order cognitive processes in Bloom’s Taxonomy. However, there are expectations for the upper-level and graduate-level courses to focus more on the higher-level of cognitive processes.  This expectation was clearly spelled out in the Upper-and Lower-Division course policy that was approved by the OSU Faculty Senate Curriculum Council in April2021.

Therefore, instead of looking at the knowledge or skills students are lacking, try thinking along the following lines to stretch the course learning outcomes into the higher-level thinking processes:

  • What can the students do after they have met the lower-level portion of the learning outcomes?
  • What if all my students scored “A” in their prerequisite course(s) or they are so academically prepared to take the course?
  • What if all my students had mastered the lower-level skills required and they liked problems that are more challenging?
  • How can my students apply the skills and knowledge from the course to their professional work?

When courses aim for the higher-order cognitive domains in their learning outcomes, it inadvertently drives the assessment away from the traditional factual memorization type of assessment that is generally entailed mainly in the ‘remembering’ and ‘understanding’ cognitive realms.

As an illustration, we will use the following learning outcome as a baseline:

“Describe the winemaking process”

A glass of red wine and bunch of grapes in low light.
Image source: “Red Wine” by leguico is licensed under CC BY-NC-ND 2.0

Describe is a verb under the ‘understanding’ cognitive process.  While it is good that students are able to describe the wine-making process, think again about the greater purpose of them being able to describe the wine-making process. Why do they need to be able to describe the winemaking process? Is it so that they can recommend the best process, compare and contrast various processes, evaluate the suitability of a process, etc?

The following are some of the suggested revisions to the above learning outcome that are geared towards the higher-order thinking in Bloom’s and how it impacts the ways students are being assessed.

  1. Recommend a suitable winemaking process to produce a product with industry-accepted specifications.
    • Recommend falls under the ‘evaluating’ cognitive domain. In the assessment, the instructor can give a list of specifications of the final product (e.g., the color, purity, turbidity of the wine) and ask the students to recommend a suitable winemaking process. In the quest of selecting and recommending the suitable process, the students are exercising critical thinking skills and potentially problem-solving skills, especially if they need to suggest certain optimization in the process to produce the product with the right specifications.
  2. Evaluate the feasibility of a certain wine-making process under specified conditions.
    • For the assignment, the instructor can present a case study of company X that wants to do a start-up business in making wines. Given the specified capitals, resources, and expected lead time for the product, the students need to evaluate if the winemaking process in question is feasible. By justifying their yes or no answer, they are practicing the synthesis, reasoning, and argumentative skills which fall under the higher-order thinking process.
  3. Compare and contrast the different winemaking process commonly used in the industry.
    • Compare and contrast are within the ‘analyzing’ and ‘evaluating’ cognitive domains. To measure this learning outcome, the instructor can ask the students to compare and contrast the processes A, B, and C. The instructor can also award points if the group is able to illustrate their explanations with the aid of diagrams. By comparing, contrasting, and illustrating the different winemaking processes, the students are having a more in-depth analysis of each process and how they are being similar or different from one another.

So, the next time you are thinking about the course learning outcomes, you may want to picture them in the context of their entirety and see them in the grand scheme of things; just like how you would have seen the far distance after turning that car’s high-beam on! 

Half a century has passed since educational psychologist Robert Gagné published his influential book, The Conditions of Learning, but his ideas are relevant to online and hybrid learning even today. He presciently wrote, “The real point to be made is that use of a variety of instructional modes is both feasible and potentially effective. . . . What is needed in each case is thoughtful design and management of the learning environment.”

chalkboard, apple, eraser, iPad that says "nine events"Gagné popularized the concept of nine “instructional events.” Each represents a step in the teaching and learning process:

  1. Gain learners’ attention
  2. Inform students of learning objective
  3. Stimulate recall of prior knowledge
  4. Present stimulus material (content)
  5. Provide learning guidance
  6. Elicit performance (practice)
  7. Provide feedback
  8. Assess performance
  9. Enhance retention and transfer

Do you regularly use these steps in your courses? To address this question, you might consider what your students experience as they work through a typical weekly module in one of your Canvas course sites:

     Are they aware of weekly learning outcomes? If not, state them in a weekly overview that serves as the first page in each Canvas module!

     How do you gain their attention? Through video, audio, images, interactivity?

     Do you use your students’ prior knowledge as a scaffold for the learning in your course? In fact, do you make it a point to assess what they know at the outset your course?

     Beyond simply presenting content, how do you provide substantial learning guidance? Do you foster active learning through online interaction student-to-student, student-to-content and student-to-instructor?

     Do you frequently assess performance and provide feedback? Do you use weekly quizzes and/or low-stakes writing or problem-solving exercises?

Ultimately, over the course of a term, have you blazed a trail that fully supports your students’ retention and transfer? If so, bravo!

Additional Resources:

  1. Gagné’s 9 Events Applied to All Courses
  2. Instructional Strategies for Online Classes

And, for a light-hearted view of teaching and learning, here’s a 3-minute video that will definitely gain your attention and may well enhance your retention of this post: Gagné’s 9 Events Featuring Cats

outcomeswordleOSU’s Curricular Policies and Procedures specify that every course syllabus should include measurable student learning outcomes.  The outcomes are defined as “learner-focused statements reflecting what a student will be able to do as a result of an instructional activity. Each outcome statement should start with a measurable action verb that indicates the level of learning, followed by a precise description of the learned behavior, knowledge, or attitude.”

For guidance in developing learning outcomes, educators have long turned to Bloom’s Taxonomy, with its pyramid of cognitive levels.  Two much newer tools can help you refine learning outcomes for a course or find learning activities and associated tech tools that will align with your course outcomes.

1 – On the Ecampus website, the interactive Objectives Builder, created by James Basore, is a wonderful tool to assist in writing learning outcomes.  It’s easy to use and can do wonders if you’re grappling with learning-outcome-writer’s block!

2 – Allen Carrington’s Padagogy Wheel has hot links to 63 iPad apps, many of which exist in forms for other mobile platforms as well.  Each app is arranged on a wheel to align with learning activities that could be done with the particular app, related action verbs, and the corresponding cognitive domains from Bloom’s Taxonomy.  Brilliant . . . give it a try!