About the Author: John Morris is Senior Instructor I in the College of Business. Prior to joining OSU in 2009, he worked at Hewlett-Packard Company in software and technology training. He was part of the OSU Blackboard to Canvas LMS (Learning Management System) transition in 2013 and 2014 and was a College of Business Peer Supporter during the Pandemic in 2020 and 2021. He is the course coordinator for BA 466 – Integrated Strategic Experience, the capstone course for business majors that is delivered by seven faculty members to about 900 students each year.
If you knew me, you would know that I have a lazy streak that disguises itself as keeping busy. I’ll work for an hour to do something if it will save me 15-minutes every quarter. My math is straightforward even if my time horizon can sometimes be a little problematic. That is, if I only employ my hack three times, I’ve technically wasted time in the process. If it feels like a repetitive task, however, I’m willing to gamble with my outlay of time.
The OSU College of Business (COB) is accredited through the Association to Advance Collegiate Schools of Business (AACSB). As part of accreditation, faculty are expected to measure specific learning outcomes to track student performance and improve teaching strategies (instruction, assignments, and course structure.) BA 466 measures student performance for four AACSB learning outcomes. While AACSB accepts sampling of student success, I had the idea that we could track performance at an individual level with less work and greater utility using the Outcomes function of Canvas.
When I took on the role of course coordinator in 2019, I consulted with our OSU Canvas guru, Tasha Biesinger, about how to use Canvas Outcomes. Tasha told me that Canvas Outcomes are a lot of work and few people at Oregon State were using them, making it somewhat uncharted territory, and something I was welcome to tackle it if I wanted. Instructure [parent company of Canvas] supplies some excellent resources on the topic including, What are Outcomes? and Overview: Improved Outcomes Management Feature Preview. When I presented my ideas to BA 466 faculty, they agreed to build Outcomes so we could measure our four AACSB learning outcomes collaboratively. I don’t think any of us, including me, knew the size of the task we were taking on!
Set up is done one time only and can easily be recycled using Canvas’s course copy function; run refers to the process each quarter of capturing learning outcome progress. Canvas Outcomes are added to Rubrics in a similar fashion as Criteria, but they behave differently. You can build a Criteria when creating your Rubric, but Outcomes must be built ahead of time and found when building or editing a Rubric.
Faculty grade students as they normally would for any assignment, clicking the ratings in Speedgrader for the Outcomes added to the Rubric. The Learning Mastery Gradebook must be enabled from course Settings. Enabling the feature allows you to select it in Grades and the contents downloaded to a CSV file for use in spreadsheet software like Excel or Sheets.
A critical enabler in our effort was the creation of an automated spreadsheet developed by COB Assistant Dean of Accreditation Byron Marshall, Ph.D. We copy/paste the CSV data from Canvas into an automated spreadsheet that takes the tedium out of the sorting process. The automated spreadsheet produces a matrix of numbers, sorted by section and instructor, showing the criteria and a composite result for each. Go here for a complete description of the workflow.
We set up and ran our first collective assessment in 2022Sp with four faculty members assessing 174 students. The process was replicated in 2022Fa for another two sections and 82 students. We had some stumbles along the way when some faculty didn’t fully understand the workflow. Importantly, once the course is closed by the Registrar, if the Learning Mastery Gradebook was not already enabled, faculty must impose on Canvas Support to enable it.
With data collected quarterly and faculty experimenting with different strategies to improve student outcomes, we now have robust, data-driven conversations about what works and what doesn’t to produce good student performance. We’re still fine tuning the process and we expect that improving our teaching will never end. Although we’re a long way from transforming teaching from art to science, now our conversations about student performance also includes student-level, venue- and modality-specific data.
As for my lazy streak, I spent more than an hour (obviously) working out this process. But now that we have a process, me and my colleagues in BA 466 have turned a 2 to 3-hour task done annually into a 5-minute task done quarterly. Since this is a task with a long-term horizon, this was one effort with a high return-on-investment!
For Oregon State University Canvas resources and support, see Learn@Oregonstate.edu
For more information about learning outcomes, see the OSU Student Learning Outcomes Policy and the Student Learning Outcomes page
Leave a Reply