by Chris Gasser

As many of you know, Supplemental Instruction (SI) is an academic support program that offers peer-led, collaborative, group, study tables for historically challenging courses. In Fall 2021, SI underwent a significant expansion, which allowed us to do some pretty cool analyses, focusing on students who were retaking courses.

Context

A common criticism of SI data is that the program is based on an opt-in model, which means that our impact data is always subject to self-selection bias. The criticism suggests that it is possible that only the most successful students may opt in to SI, skewing impact data in favor of SI. In an effort to better understand SI’s impact while trying to account for this possibility, I performed an analysis of students retaking courses. By focusing only on a retaking population, I hoped to minimize any effect from the most successful students, assuming that retaking a course demonstrated a certain level of academic difficulty.

Methodology

For all SI supported courses in AY 21-22, I pulled the course grades for all students enrolled in the courses. I then pulled the most recent course grade for anyone who had previously taken the course at OSU and received a course grade. This gave me a list of 1,949 students who were retaking courses that SI supported in AY 21-22. Of those 1949, 1382 had previously earned an A-F course grade, and 567 had previously earned a W grade. I focused on these two populations in the analysis.

Results

Here are a few of the key results from the analysis of students retaking courses.

Among students who previously earned a course grade, students who completed SI earned a higher course grade than students who did not participate in SI.

When looking at the first grade that students earned, both students who later completed SI (attended 4+ times) and students who did not use SI earned the same course average of .71 in their previous attempt at the class. This was a good indication that there was similarity between the groups. When looking at their subsequent grades, students who did not participate in SI earned a new average of 1.77. For students who did complete SI, their new average was 2.29—a difference of .52 grade points, or a half of a grade point higher than the average for students who did not participate in SI.

SI made a meaningful impact on the number of students passing the course

While SI helps students earn higher course grades, the boxplots tell an interesting story: in both analyses the SI 4+ 25% quartile ends at 1.7 (C-), which is the mean of the No SI groups. Not only do we see students in SI earning higher average (mean and median) course grades, we also see a much lower number of students below that 1.7 threshold. This provides evidence that SI both helps students earn higher course grades and increases the number of students who pass the class.

Among students who previously earned a W, students who completed SI earned a higher course grade than students who did not participate in SI

A common idea on campus is that students who earn a W grade should not be included in analyses with students who earn a D/F, because students may withdraw from a course for reasons outside of academic difficulty. Separating this group out, we still see a similar effect in mean course grade: for students who completed SI, an average grade increase of .49 higher than students who did not participate.

Discussion/ Conclusions

Focusing on this narrow population allows us to compare data among students who may have more in common while also avoiding any skewing effect of our highest achieving students. What we see is strong evidence for the efficacy of the SI program, especially for students retaking courses. The effect on students’ course grades was even more pronounced in this analysis than a separate analysis for students not retaking the course, which prompts me to think more about how we get students retaking courses in to SI early. It also makes me wonder if we might better support our students retaking courses through building out a more robust support structure (of which, SI could be a part). There is more to this report than I can share here, and I would love to chat with you in more detail about my process and findings. If you’re interested in hearing more, interested in new ways to assess student success, or interested in thinking about how we better support our retaking students, please reach out. I’d love to chat (chris.gasser@oregonstate.edu).