The following is a guest blog post from Meilianty Gunawan. Meili completed an Instructional Design internship with OSU Ecampus during Fall 2021.
Have you ever driven a car on a highway with no streetlights in the middle of the night? Your first instinct is to turn your car’s high-beam lights on to give you greater visibility on what lies far ahead of you. You are probably fine just by relying on the car’s low-beam lights, but you will not be able to get a clearer picture of the far end of the road as you would get from the high-beam.
Bloom’s Taxonomy is a framework used by instructors to develop course learning outcomes. It lays out six cognitive domains (from basic to more advanced) and examples of measurable action verbs along with those domains.
When course learning outcomes are set from the point of students’ knowledge and skills deficiency, students are inadvertently deprived of the greater things that they were able to achieve after completing the course. Lower-division courses (100- and 200-level courses) generally focus on the lower-order cognitive processes in Bloom’s Taxonomy. However, there are expectations for the upper-level and graduate-level courses to focus more on the higher-level of cognitive processes. This expectation was clearly spelled out in the Upper-and Lower-Division course policy that was approved by the OSU Faculty Senate Curriculum Council in April2021.
Therefore, instead of looking at the knowledge or skills students are lacking, try thinking along the following lines to stretch the course learning outcomes into the higher-level thinking processes:
What can the students do after they have met the lower-level portion of the learning outcomes?
What if all my students scored “A” in their prerequisite course(s) or they are so academically prepared to take the course?
What if all my students had mastered the lower-level skills required and they liked problems that are more challenging?
How can my students apply the skills and knowledge from the course to their professional work?
When courses aim for the higher-order cognitive domains in their learning outcomes, it inadvertently drives the assessment away from the traditional factual memorization type of assessment that is generally entailed mainly in the ‘remembering’ and ‘understanding’ cognitive realms.
As an illustration, we will use the following learning outcome as a baseline:
Describe is a verb under the ‘understanding’ cognitive process. While it is good that students are able to describe the wine-making process, think again about the greater purpose of them being able to describe the wine-making process. Why do they need to be able to describe the winemaking process? Is it so that they can recommend the best process, compare and contrast various processes, evaluate the suitability of a process, etc?
The following are some of the suggested revisions to the above learning outcome that are geared towards the higher-order thinking in Bloom’s and how it impacts the ways students are being assessed.
Recommend a suitable winemaking process to produce a product with industry-accepted specifications.
Recommend falls under the ‘evaluating’ cognitive domain. In the assessment, the instructor can give a list of specifications of the final product (e.g., the color, purity, turbidity of the wine) and ask the students to recommend a suitable winemaking process. In the quest of selecting and recommending the suitable process, the students are exercising critical thinking skills and potentially problem-solving skills, especially if they need to suggest certain optimization in the process to produce the product with the right specifications.
Evaluate the feasibility of a certain wine-making process under specified conditions.
For the assignment, the instructor can present a case study of company X that wants to do a start-up business in making wines. Given the specified capitals, resources, and expected lead time for the product, the students need to evaluate if the winemaking process in question is feasible. By justifying their yes or no answer, they are practicing the synthesis, reasoning, and argumentative skills which fall under the higher-order thinking process.
Compare and contrast the different winemaking process commonly used in the industry.
Compare and contrast are within the ‘analyzing’ and ‘evaluating’ cognitive domains. To measure this learning outcome, the instructor can ask the students to compare and contrast the processes A, B, and C. The instructor can also award points if the group is able to illustrate their explanations with the aid of diagrams. By comparing, contrasting, and illustrating the different winemaking processes, the students are having a more in-depth analysis of each process and how they are being similar or different from one another.
So, the next time you are thinking about the course learning outcomes, you may want to picture them in the context of their entirety and see them in the grand scheme of things; just like how you would have seen the far distance after turning that car’s high-beam on!
In your work as an instructor or instructional designer, what kinds of course design problems have you been trying to solve lately? Perhaps there’s a discussion board assignment that you’d like to make more engaging, or maybe there’s a lab activity that students struggle with.
According to recent research findings published in the journal Nature, “…people are more likely to consider solutions that add features than solutions that remove them, even when removing features is more efficient” (Meyvis, T. and Yoon, H., 2021). In a course design context, this means we will probably reach for something we can add to our course to fix a problem: a shiny new tool we learned about at a conference, an extra video lecture, or an additional step for that lab activity that students struggle with. We do need to include quality tools and fresh media to our courses, certainly. However, because there is so much to cover in any given course, we should strategically subtract all unnecessary elements. The aforementioned Nature article has brought to my attention that I may be regularly neglecting the powerful design move that subtraction offers. As a result of reflecting on that, I’ve collected a few examples of how instructors and instructional designers can use subtraction in course design.
Subtraction example #1: Remove formal requirements from informal practice activities
Let’s take a look at a discussion board assignment as an example. Sometimes discussion boards are treated more like formal writing opportunities than discussions. For example, do you require students to not only respond to the prompt but also use a formal style of discourse and cite evidence in accordance with APA style guidelines and do so in a certain number of words? We don’t require students to use APA style or converse with a word count in our in-person discussions, so why would we do this in an online course (Darby, 2020)? Consider subtracting those formal writing requirements for discussion boards, and reserve the APA citations and formal requirements for polished writing assignments. That way, online discussions can be a place to practice and informally grapple with new concepts and ideas the way that face-to-face discussions are. Then, after students have a chance to work through topics in an informal way, they will be ready to try more formal tasks on the topic, such as constructing an argument and appropriately citing evidence according to a particular style guide.
Subtraction example #2: Reduce the frequency of office hours
If you hold office hours at a set time each week, and you notice that few students attend, consider doing away with the set weekly “office hours” altogether. Instead, invite students to make an appointment with you at least once during the term, at a time and in a modality that works well for them, or reduce the frequency of office hours to two or three timely sessions (optional attendance) per term, such as before each exam. Take questions in advance so that students too busy to attend can still benefit from the sessions, and post a recording of the session afterward in an announcement. According to Lowenthal, Dunlap, and Snelson (2017), less frequent but more focused office hours increased participation from students. Lastly, consider renaming your new infrequent office hours to something warm and inviting, such as “Coffee Break” or “Consultations” (Darby, 2019), which students may find more welcoming. Building in plans for facilitation and instructor presence in the early stages of course development allows faculty to focus more on teaching while the course is running and less on reactive problem solving.
Subtraction example #3: Dethrone teaching “folklore”
Teaching “folklore,” which John Warner calls the ineffective “practices handed down instructor to instructor” (p. 207, 2020), shows up uninvited, particularly when you have been assigned a course to teach that you had no hand in developing. One example of teaching folklore is the stubborn assumption that serious scholars are gatekeepers for their fields. Evidence of this exclusionary approach may show up in the form of a stern, overly formal, or cold tone in a syllabus. Consider removing verbiage that conveys a cold tone, since we now know that warm-tone syllabi encourage students to reach out to their professors (Gurung and Galardi, 2021). Another example of this is inflexible class “policies” that reflect an individual instructor’s preferences and not university policies. Consider reviewing the policies stated in the syllabus and delete any that are not aligned with actual university policies. Further, the idea that students must achieve a level of eloquence and scholarly sophistication on par with faculty in order to be considered for a grade of “A” on an assignment is another example of this. Evidence of this type of folklore could be found in rubrics with benchmarks that are nearly impossible to achieve. The element that could be subtracted here is not the rubric, but rather the specific language in the rubric that makes it impossible for students to succeed in the assignment. Neuromyths, which are false beliefs about the brain and learning, could also be included in the category of teaching folklore. If you spot neuromyths in a course, remove them. In summary, if you spot one of these ineffective teaching folklore elements in your course, consider removing the “folklore” item.
For this design challenge, try subtracting ineffective design elements before adding new items to solve course design problems. If you are unsure if something should stay or go, ask yourself what purpose this element serves in the course. It should then become clear whether the item belongs or needs adjustment. Even small adjustments can transform learners’ experiences. What have you removed from your course? Share how the process was for you by leaving a comment.
References:
Blum, S. D. (Ed.). (2020). Ungrading : Why rating students undermines learning (and what to do instead). West Virginia University Press.
Darby, Flower, and James M. Lang. Small Teaching Online : Applying Learning Science in Online Classes, John Wiley & Sons, Incorporated, 2019. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/osu/detail.action?docID=5780349.
Created from osu on 2022-01-18 23:50:41.
Gasiewski, J.A., Eagan, M.K., Garcia, G.A. et al. From Gatekeeping to Engagement: A Multicontextual, Mixed Method Study of Student Academic Engagement in Introductory STEM Courses. Res High Educ 53, 229–261 (2012). https://doi.org/10.1007/s11162-011-9247-y
Gurung, R. A. R., & Galardi, N. R. (2021). Syllabus Tone, More Than Mental Health Statements, Influence Intentions to Seek Help. Teaching of Psychology. https://doi.org/10.1177/0098628321994632
Lowenthal, P. R., Dunlap, J. C., & Snelson, C. (2017). Live synchronous web meetings in asynchronous online courses: Reconceptualizing virtual office hours. Online Learning , 21 (4). https://doi.org/10
Relationships matter for successful collaborative work. Yet, when it comes to online/ blended/hybrid course design, development often begins with a focus on course content, assuming that the collaborative relationship between faculty and instructional designers is secondary to the design process (Tate, 2017). If we care about student success, we must turn our attention to the ways in which effective collaborative relationships among design partners contribute to the course feel — that is, the online learning environment as perceived by students. In our recent OLC presentation, we proposed an interactive strategy for developing collaborative relationships between faculty and instructional designers based on curiosity, enthusiasm, and mutual respect.
Faculty and instructional designers often find themselves at cross-purposes. The design process expects two groups of experts to bring their unique perspectives and skill sets together in course creation, without providing instruction or support for the kinds of intentional shared knowledge transmission and production necessary for collaboration (Richardson, J. C., et al., 2018). In fact, faculty buy-in to a collaborative working relationship ranks as the number one obstacle to instructional designers’ success (Intentional Futures, 2016, pp. 3, 15). Institutional frameworks must be in place that set up faculty-instructional design teams for success, so that together they in turn can set up students for success.
We suggest that fostering curiosity in each other’s disciplinary norms and approaches serves as an igniting spark for establishing effective instructional designer-faculty autonomy supports. In this way, relationships begin with trust, mutual respect for professionals’ expertise, and socioemotional growth.
ID-Instructor Cooperative Instrument
To aid in this process, we developed an ID-Instructor Cooperative Instrument for individual, flexible needs of both parties. Through a series of definitions and short prompts, users can see similarities and differences between their viewpoints on topics like student success, well-designed courses, and course feel. Feel free to use the spreadsheet linked above or this list of questions:
Course feel: Name up to 5 keywords that describe how you want the course to feel.
Student interactions: List the kinds of interactions your students will encounter in your course. Feel free to list them in order of importance, or to modify the categories.
Teacher ⟷ Student
Content ⟷ Student
Student ⟷ Student
Definitions:
How do you want students to define success in this course?
How do you define success in this course (for yourself and/or students)?
Working Together:
Name your best course — one you have designed, taught, taken, etc.
Name a course activity you are proud of, or one you would like to be proud(er) of.
Identify 3-5 keywords or phrases that describe your working style.
Anything else you would like to note?
Initial Feedback
In our OLC presentation, some session attendees were purely instructional faculty, while others worked full-time in instructional design. Many people served in joint roles, including some administrative responsibilities for facilitating course design, or had started their careers in one area and then switched roles. Despite these different backgrounds, almost everyone agreed with our assertion that lack of mutual respect between faculty and instructional designers negatively impacted their ability to create and deliver high-quality courses.
During the presentation, we enjoyed seeing the enthusiasm and excitement from participants wanting to share their own stories and experiences. Working through the instrument questions as a group for this session was engaging for both participants and for us as presenters. Participants were able to quickly and clearly pick up the instrument and begin to share their ideas right away. Feedback both during the interactive presentation and during the Q&A suggests that our instrument serves the purpose we created it for: to get faculty and IDs excited about talking with one another about teaching and course design.
Conclusion & Getting Started
We advise other faculty and ID teams to rethink their working relationships, starting with curiosity about each other’s experiences and hopes for the course and their future students. Administrators — who often need to increase faculty buy-in to course development programs, and help instructional designers meet faculty where they are in their pedagogical experience and comfort levels in online instruction and design — can play an important role in encouraging design teams to take the time to work through the kinds of questions and conversations outlined in this tool. The road to pedagogical expertise is often varied, windy, and complex, for everyone involved. When the course design process focuses on growth and learning for the faculty and ID, it leaves a positive mark on a course that reverberates for the students experiencing that online space.
We invite faculty and IDs to get curious about their counterparts. If there is not currently an intake meeting for faculty and IDs at your institution, you could share this instrument either for a one-on-one meeting, or with a dedicated pedagogy session through your Center for Teaching and Learning and discuss how it could be a starting point for a collaborative working relationship. If there is already an intake meeting that is part of the course design process, think about bringing in aspects of this instrument that might be missing from that session.
If you are an administrator, suggest that your faculty or IDs spend some time at the beginning of a project getting to know one another. You might even help spark some curiosity by including a professional introduction — what excites you about having a particular instructional designer or faculty member working on a specific course or program? What can you share about the unique experiences of your faculty or IDs that would help start that initial conversation?
We hope that this instrument will be a tool you can use as new course developments begin, whether you are working with a new collaborator or wish to get to know someone better that you have worked with previously. We would also be interested in continuing to learn about faculty and instructional designer relationships. If you have feedback, comments, or experiences you would like to share, we invite you to leave a comment on this blog post or reach out to us via email.
Subject matter experts in many fields have embarked on authoring projects with the goal of replacing traditional published texts or customizing content for specific learners’ needs, yet large-scale creation of open textbooks or series designed for language learners has been slower to gain traction (Blyth and Thoms, 2021). Efforts in this area have largely been limited to adapting existing OER materials (5R activities) for specific learning contexts or piecemeal creation of online activities to provide reinforcement of isolated language skills. Part 1 in this series outlines the potential benefits, limitations, and challenges that programs and instructors face when undertaking large-scale authoring projects to address the needs of language learners. The purpose of this second post is to offer guidance for creating open source language texts and present a framework for getting started.
Language Acquisition as an OER Subject Matter
Before getting into the nitty-gritty of creating an OER for language learners, it is worth pointing out how this process differs from open authoring projects for other disciplines. While OER writing projects come with inherent challenges regardless of the field, authoring comprehensive language learning content presents a unique challenge. One reason for this is that language teaching and acquisition involves complex sequencing and scaffolding of skills, language items, and linguistic concepts unique to the field of language acquisition (Howard and Major, 2004). An effective resource must present language items not only in the established order of second language acquisition, but also at the correct level for the learners at hand. The “correct level” is fluid and influenced by many variables (first language interference, motivation, metacognitive skills related to the process of learning a language, fossilization of rules, literacy in the first language, prior knowledge and educational experiences, and so on). While, say, writing a history text also involves expert scaffolding to ensure that content builds on what came before, there are fewer moving parts to align, and presentation and sequencing can and will vary from one subject matter expert author to the next. Effective language learning materials, on the other hand, are more like a house of cards that relies on complex relationships between a variety of aspects of the target language, language input, learning context, and characteristics of the learners themselves.
Language acquisition occurs when input is just beyond what students understand of a language (known as comprehensible input i + 1), so writing materials that consistently hit the sweet spot for learning is tricky even for the most seasoned language educator. In addition to presenting language in a specific sequence and at the appropriate level, authors must also consider how much new language content is enough at each stage and how to introduce, recycle, and reinforce this language through engaging and original texts as well as audios that present authentic and relevant contexts. All of this new content must then be aligned to learning outcomes related to both language form and function. Learners need not only the nuts and bolts of the language, but pragmatics are equally important—how is the language used in specific contexts and with a variety of interlocutors? What models will convey this information accurately in a way that is accessible at different levels of proficiency?
Understanding the social aspect of language is as important as the grammar and structure. Language proficiency involves much more than speaking and listening in a new language. All of the linguistic aspects must be delivered via content that also serves as a vehicle for familiarizing learners with the cultural and social contexts where the language is spoken. This should be done in a thematic way, weaving in social justice issues in a timely and relevant manner, sensitive to the complexity of the issues at hand. The author(s) must also be able to write engaging texts that present level-appropriate target grammar, vocabulary, and cultural information in activities that build upon each other or recycle the language previously taught. In addition to being linguistically sound, these original texts and audios must also demonstrate awareness and care for representation, integrating cultural and social justice topics relevant to the diversity of cultures, communities, interests, and social issues of a variety of speakers of the target language.
Creating language materials that incorporate all of these considerations requires a broad skillset beyond expertise in teaching the subject matter. Significant professional development and exploration may be necessary before embarking on expansive authoring projects for language learning. With careful coordination and planning, and an understanding of the process and support required, language programs, instructors, and learners stand to reap long-term benefits from creative, relevant, inclusive, and dynamic open resources. What follows is a suggested process and a sample framework for undertaking open source textbook production for language learning contexts.
Step 1: Survey of needs, resources, and intended uses
The first step in any authoring project is to identify the needs of a program and determine what kind of text will be used and how. This includes the extent of the textbook use within the department and also as a resource for the broader language learning community. A quality textbook resource can increase student autonomy and interest in language learning generally by providing accessible resources readily available to learners anywhere in the world (Godwin-Jones). At this stage, some questions to ask include:
Who are your learners and what is their motivation for learning a new language?
What kind of textbook needs to be replaced? What gaps do you seek to fill by replacing current text materials?
How will the resource fit into the larger curriculum?
Is the need for a single course text or a cohesive series to cover an entire level or multiple levels?
Who is available to collaborate and what are their areas of expertise?
What resources and support are available for the project?
How will time and resources be accounted for?
What is the timeline for implementation?
Who will provide expert and outside peer review of the textbook materials?
How will the textbook materials be maintained and updated over time to ensure long-term viability?
Most language programs involve extensive faculty collaboration. Instructors teach multiple courses across various levels. As such, it is important to promote broad participation in the planning stage to encourage instructor input, address concerns, and determine the scope of the implementation (e.g., across courses, levels). Gathering input on content and soliciting expertise among colleagues increases faculty buy-in and ownership of new materials.
Step 2: Identify the scope of your project
Once the shape of the project has been determined, it is time to outline the specifics of the resource(s) to be created. Determining the scope involves identifying learning outcomes. These are often mandated by the program, but they might also need to be rewritten or revised by the authoring team. Both the Common European Framework of Reference for Languages (CEFR) and the American Council on the Teaching of Foreign Languages (ACTFL) provide guidance on proficiency standards for world languages. Beyond language outcomes, it is also important to identify what the content goals are. This may involve generating a master list of topics, social justice and cultural themes to be addressed, and so on. At this stage, identify subject matter experts and any professional development needs. Who will participate as part of the core authoring team? Who will be responsible for quality control and review? Keep in mind that creating, reviewing, building, piloting, and maintaining comprehensive open language materials requires a significant time commitment, even if the goal is to create a single course text.
Step 3: Collaborate on a framework scope and sequence
The importance of this stage cannot be understated, particularly when multiple authors and stakeholders are involved. Once the themes, topics, outcomes, and other aspects of the materials have been identified and divided up into learning segments (levels across a series, units of a text, etc.), it is time to write the scope and sequence, which is basically a detailed outline. This outline should clearly show how everything fits together and serves as a framework for authoring materials that weave together language function and form, grammar points, text genres, readings, audios, pronunciation, vocabulary, practice activities, interaction, cultural and social justice themes, topics, etc.—all with attention to a logical progression, order of acquisition, recycling of language items, and student engagement.
Let’s take a language text for English language learners as an illustrative example. If the grammar point for a learning segment is the present perfecttense, what types of contextualized content elicits that structure naturally? Perhaps a short biography representative of one aspect of the culture can serve this purpose. Reading and audio texts might present a person’s life experience so far. This offers an opportunity to consider whose lived experience will be represented via this text. Which vocabulary items are crucial for using the language around this topic? How much vocabulary will be new and how much recycled? Using the present perfect form as an example, we might focus on a number of participial adjectives and the prepositions for/since (She has been interested in … since…). Language teachers as subject matter experts should have no trouble identifying these items regardless of the target language—this is how SLA comes into play. They teach this content all the time. The challenge is in putting it all together across units. That is, in addition to the grammar, punctuation, pronunciation, and writing systems (depending on the writing system of the language at hand) authors need to consider how the language builds on what was previously taught, how much is enough to present in each segment, what the cultural and social topics to be woven into authentic texts are, etc. A good scope and sequence defines all of this. For this reason, settling on a solid scope and sequence will require extensive coordination across subject matter experts, departments (in many cases), and in all cases, several rounds of review and revision before any writing begins.
Fig. 1 presents a sample unit from a scope and sequence created for a text for learners of English (CEFR A1+ level). This structure is applicable regardless of the target language although there may be some differences depending on the characteristics of the language. For example, phonetic languages may require less focus on pronunciation while languages using non-roman script will need to build in instruction on writing systems. View an accessible version of this template.
Step 4: Write and build
Once a final scope and sequence is in place and an authoring team has been identified, it’s time to start creating materials. Some questions to guide this stage include:
Who will write the text and audio scripts?
Who will provide diverse voice talent for audio texts?
Who will record and edit audio?
Where will the content be hosted and who will create it? (If using a tool such as Pressbooks, consider training for materials developers.)
What platforms will be used for any interactive content? (H5P, Quizlet, Playposit, etc.) and who will build these?
Where will the interactive repository be stored?
Who is responsible for acquiring Creative Commons images and maps?
How will accessibility be ensured?
What learner analytics will be gathered and how?
Because most of the activities in a learning segment tend to spring from the reading and listening input, it may be helpful to start with writing all of the texts for a level or course before creating the rest of the content. Audio and written texts must expose learners to a wide range of genres and text types (Tomlinson, 2012), so it may be useful to start by generating a list of genres to be covered. The texts, whether they are original or curated from online sources, must be reviewed and revised or adapted to ensure they contain the necessary language at the appropriate level, generate interest, employ the intended tone and voice for the genre, follow identified themes, incorporate social justice and cultural topics, and are accurate (in the case of non-fiction texts). If the content is curated, it needs to be reviewed for copyright and accessibility.
Throughout the authoring stage, frequent check-ins among authors and reviewers can help to ensure quality, authenticity, inclusivity, and adherence to the determined scope and sequence.
Step 5: Review and revise
Just as at every other stage of the process, peer reviews and revisions should be coordinated so that there is continuity in the editing process. It is important to enlist the help of internal and external subject matter experts. Assign different review tasks (big picture reviews of content and continuity along with detailed reviews of the language presentation) and provide each with review rubrics or guidelines to streamline feedback. External reviewers outside of the organization can help provide neutral insight. Enlisting the help of those with expertise in social justice topics, regional cultural perspectives, and different varieties of the language can help ensure accuracy, representation, inclusivity, and engagement. Be sure to review feedback as a team to reach an agreement on how to approach revisions to draft materials.
Here is a sample rubric for reviewing original course materials, but rubrics should be adapted according to the scope and sequence and goals of each project.
Step 6: Implement and iterate
Finally, it is time to pilot your new OER, but you aren’t done yet! Deliver the content to learners and collect their feedback as well as input from your teaching team. Keep in mind that an OER can be a work in progress, and one advantage of an open textbook is that it can be an evolving resource. Each iteration should involve coordination and input from the team who will be using the materials. Share your new resource widely. A high-quality OER opens the door for resource sharing with a broad community of colleagues, building visibility for the language program and lending credibility among colleagues around the world for the subject matter expert creators (Blyth and Thoms, 2021).
Step 7: Create a plan for long-term viability: updating materials, quality control, and access
While creating an open source textbook for language learners can be a continually iterative process, once the new materials reach a point of stability and all stakeholders are satisfied with the product, authors or departments need to create a plan for maintaining the materials. Unlike costly textbooks which quickly become outdated, open access resources are easier to update (once the initial investment in time and resources has been made), save students money, and expand access to learners everywhere. The key is to create a plan for longevity so that updates are systematic, incorporate learner and instructor input, and are reviewed for quality control. The beauty of an OER for language learning is that it offers the opportunity to democratize teaching and learning by being responsive to the changing landscape of social justice education, shifting cultural influences, evolving characteristics of language learners, and distinct learning contexts.
Blyth, C. S., & Thoms, J. J. (Eds.). (2021). Open education and second language learning and teaching: The rise of a new knowledge ecology. Multilingual Matters. https://www.multilingual-matters.com/page/detail/?k=9781800411005
Godwin-Jones, Robert. “OER Use in Intermediate Language Instruction: A Case Study.” CALL in a Climate of Change: Adapting to Turbulent Global Conditions – Short Papers from EUROCALL 2017, 2017, pp. 128–134., https://doi.org/10.14705/rpnet.2017.eurocall2017.701.
Howard, Jocelyn & Major, Jae. (2004). Guidelines for Designing Effective English Language Teaching Materials.
Tomlinson, B. (2012). Materials development for language learning and teaching. Language Teaching, 45(2), 143–179. https://doi.org/10.1017/S0261444811000528
A group of instructional designers at Ecampus participated in a book club reading “Ungrading” (Kohn & Blum, 2020). We learned many creative ways of designing assessments through participation in this book club. If you happen to be searching for ideas on designing or re-designing assessments in your teaching, we would highly recommend this book!
The idea of “Ungrading” may sound radical to many of us. Yet instructors at all types of educational institutions have tried ungrading in many different courses, ranging from humanity courses, to STEM courses, and from primary education to higher education. Starr Sackstein (author of Chapter 4 “Shifting the Grading Mindset” of the book) encourages educators to consider “ways to adjust small things in the classroom that will lead to important growth for students”. And this suggestion of starting small is coherent with what James Lang proposes in his book “Small Teaching” (Lang, 2016) and Thomas Tobin’s +1 strategy for implementing new teaching and learning strategies (Tobin & Behling, 2018). Sackstein provides a table comparing the grades vocabulary that focuses on judgement or criticism, with the non-grade vocabulary focusing on assessing and opportunity for improvement.
In chapter 5, Arthur Chiaravalli proposed a way for teaching without grades: Descriptive Grading Criteria, such as A for outstanding, B for Good, C for Satisfactory and I for Incomplete. Do you remember elementary school report cards that use E for Excellent, S for Satisfactory, and NI for Need Improvement type of categories? I think that is exactly what descriptive grading criteria represent.
In chapter 7, Christina Katopodis and Cathy Davidson offer a new approach to start a new term/semester by asking students:” What is Success in this class for you? And How can I help you achieve it?” (p. 107) Katopodis and Davidson also remind us the importance of explaining why when you challenge your students to take their own learning seriously and give students opportunities for metacognitive reflections about the learning activities themselves. Katopodis and Davidson also offer a model of contract grading for Twenty-First Century Literacies and a model of collaborative peer evaluation. Students’ grades in the course come from self-and-peer evaluations using detailed evaluation forms.
In chapter 8, Christopher Riesbeck described his critique-driven learning and assessment design of do-review-redo submission process for his intermediate-level programming course. I have used similar approach in my own teaching before and it works very well for any course with manageable number of students. The advantage for this approach is every one of your students can improve their first submissions based on feedback they receive from the instructor. The disadvantage for this approach is the potentially extended time instructors may spend on providing the feedback and reviewing the submissions and re-submissions. The key to this assessment method is making sure that the workload of providing feedback and reviewing revisions is manageable. In chapter 9, Clarissa Sorensen-Unruh provided her experience of using ungrading in her organic chemistry II course, giving students opportunities to practice evaluating their own work.
And that is only snippets of what I took away from a few chapters from this book. Many resources about ungrading outside the book were shared during our book club meetings, such as two-stage exams, group exams and public exams. To answer a common question that ungrading practices may fit humanity courses more easily, Cyndie McCarley shared “Grading for Growth” blog written and maintained by two math instructors Robert Talbert and David Clark. To learn about all the creative assessment design methods introduced in this book, read it yourself either through library ebook or get a hard copy and enjoy reading, designing and experimenting!
References
Kohn, A. and Blum, S. (2020). Ungrading. West Virginia University Press.
Lang, J. (2016). Small Teaching. Jossey-Bass.
Tobin, T.J. and Behling, K.T. (2018). Reach Everyone, Teach Everyone. West Virginia University Press.
At a recent faculty professional development workshop series, I became aware of faculty’s concerns about addressing the learning needs of students from different cultural and linguistic backgrounds. Students with multilingual and multicultural identities are oftentimes perceived as deficient (Almon, 2014; Flores & Rosa, 2015) and might feel they hold an outsider status (Merryfield, 2000). In my personal experience navigating multiple identities that intersect culture and language, and in my work supporting faculty in their learning design and instructional decisions, I began examining ways in which blended and online learning spaces can offer more welcoming opportunities for students. One of these ways is using a cultural lens and mindset towards inclusive learning design.
Culturally Responsive Approaches
There have been several culturally responsive approaches to teaching and learning. By and large these approaches advocate for the recognition of students’ cultural backgrounds as critical to their learning success (Gay, 2013; Ladson-Bilings, 1994). In fact, a culturally responsive pedagogy (CRP) aims to promote the integration of students’ culture to support their learning experiences. In blended and online learning, this pedagogy can create an environment that “acknowledges, celebrates, and builds upon the cultural capital that learners and teachers bring to the online classroom” (Woodley, Hernandez, Parra, & Nagash, 2017, p.1).
For students whose first language is not English, we first should focus on their strengths and not their deficiencies. These students bring their cultural backgrounds, values, experiences, and language diversity with them to the online learning environment. It is important to recognize that culture is central to teaching and learning; therefore, advancing online and blended learning design should be grounded on dimensions for cultural sensitivity where students’ diverse identities, cultures, languages, and backgrounds are seen through an asset-based lens. This means, recognizing the value in the cultural backgrounds, experiences, and languages of students; and embracing these students’ traits as assets. This asset-base approach can be the first step in developing a mindset for designing and teaching in ways that promote social, academic, and emotional learning for these and ALL students.
Culturally responsive approaches have been mapped out to the three principles of Universal Design for Learning to offer (1) multiple means of engagement, (2) multiple means of representation, and (3) multiple means of action and expression (UDL, n.d.). UDL and CRP can help instructors amplify the opportunities for students from different cultural backgrounds to demonstrate their knowledge when given strategies that incorporate multiple perspectives, experiences, connections to the real world, and choices (Bass & Lawrence-Riddell, 2020; Kieran & Anderson, 2018)
The connection of UDL and CRP offers consideration to inform instructional design choices. Yet, these considerations appear to be adds-on to the design of the learning experience. How can we expand the UDL and CRP connection to embrace a mindset to move towards an inclusive learning design where the cultural and linguistic traits of students are seen from an asset-based perspective? A few dimensions from research and praxis would get us started to help achieve this goal.
Dimensions for Learning Design
The following dimensions for learning design, that expand the connections between UDL and CRP, should be considered whenever possible in the design of blended or online learning experiences. Following are the six dimensions.
Dimensions for Culturally Responsive Learning Design
Instructor’s reflection
Visual design
Linguistic domain
Content
Interaction
Technology
Identity and Experience
Instructor identity and experience relates to the practice reflection –inner analysis to reveal assumptions about teaching and learning (Jaramillo Cherrez & Jin, 2020). Through these reflections, instructors can identify ways to humanize the learning experience. Instructional designers (IDs) can help instructors engage in a (self) dialogue to explore how the instructor’s identity informs or impacts their teaching and instructional decisions, how they respond to students’ cultural differences and embrace them as strengths, and how instructors could also learn from students.
Design
The visual design of the course and learning materials can have a profound impact on students’ learning experience (Hedberg & Brown, 2002). For students whose first language is other than English in particular, it is important to be aware that these students may come from different cultures and social groups, and thus, visual representations may have positive or negative consequences to their success in the course. Visuals should be carefully chosen because the variety of images, colors, and symbols may affect the message students receive in the class. When using images from pop culture, it is helpful to add context to give more clarity to the instructional purpose. The visual design also relates to the readability of the content and how it is presented and structured. Asking a colleague or friend to read the instructions and descriptions of assignments can help clarify expectations and requirements for students. Bear in mind that what is clear to one is not always clear to others, especially when using complex sentences and terminology of a discipline.
Linguistic Variability
Many students might have a first language different from English. Also, keep in mind that different cultures may have different ways of writing, usually influenced by rhetorical and social contexts (Almuhailib, 2019). There may also be linguistic and cognitive differences in the way that students interpret the information given to them. For some cultures, direct descriptions are fine, whereas for others the context is important before addressing a specific perspective. Some cultures may characterize themselves for being more individualistic and others more holistic, and students, including those whose first language is other than English, can find themselves moving along that continuum. In designing culturally responsive blended or online courses, language matters because of the transactional distance characteristic of asynchronous spaces. Many students may already be pressured to demonstrate “good”, “academic”, “professional” English. One way to be aware of linguistic diversity is to be more explicit with instructions. For example, indicate clearly the use of naming conventions, abbreviations, acronyms, and descriptors in activities and assignments.
Content
The fourth consideration is content. The main suggestion is to try to diversify the curriculum with resources from around the world (e.g., content from scholars from diverse cultures and linguistic backgrounds). Allow students to see themselves represented in the materials. Create activities and assignments that help students explore the concepts in connection to their own backgrounds and communities(e.g., linguistic, cultural) and experiences, and that allow students to move from low to high cognitive tasks (e.g., staged projects). Yet, diversifying the curriculum goes far from bringing into the course content perspectives that are commonly ignored. It involves explicit acknowledgement of the value of the different perspectives and modes of knowledge.
Interaction
Interaction also can benefit from a culturally responsive mindset in that instructors can vary the modes of interaction by using audio/video communication(e.g., assignment feedback, DB, announcements). It is also important to guide and scaffold group activities with resources such as guidelines, group contracts, teamwork guidelines, group rapport activities, conflict resolution resources). Particularly for teamwork, instructors can build group activities early in the course to promote collaborative learning. For online discussions, instructors could allow students to select the tools that they feel more comfortable with using, bearing in mind that many students from different cultural backgrounds might not be familiar or have experience participating in discussion activities. Another suggestion is to promote student-led discussions to help students move from the individual task to the group task. This will allow to vary the cognitive demands that can foster meaningful knowledge construction and organization while also addressing different audiences, styles of writing and analysis, and communication modes.
Technology
It is important to recognize that technology is not neutral. Clearly, it is important to select tools and evaluate them for intended and unintended consequences for students, such as the cost, the technical support, the pedagogical affordances, and the availability in other geographic areas. It will be helpful to consider the different levels of technology skills that students may have and plan on developing guidelines and technical resources (e.g., links to providers, manuals, accessibility and privacy policies) that can help students. Additionally, in considering digital tools it is important to review whether the affordances the tools offer are available to all students, in the different browsers and devices (e.g., tablets, smart phones, browsers). In using digital tools, careful attention should be given to the kinds of data that the tools require students and instructors to share. It is important to read carefully the terms of use, data privacy, and the information that is being collected as a way to understand how the users can trust the tools and their procedures for sharing or not with others the data collected.
These dimensions underscore the need to approach learning design with a mindset that not only acknowledges student multilingual and multicultural identities, but also catalyzes these identities to help students be valued and successful. I consider these dimensions in my instructional design work, and I would like to invite you to consider them next time you design an online or blended learning experience.
Sources
Almon, C. (2015). College persistence and engagement in light of a mature English language learner (ELL) student’s voice. Community College Journal of Research and Practice, 39(5), 461-472.
Almuhailib, B. (2019). Analyzing Cross-Cultural Writing Differences using Contrastive Rhetoric: A Critical Review. Advances in Language and Literary Studies, 10(2), 102-106.
Bass, G., & Lawrence-Riddell, M. (2020). Culturally Responsive Teaching and UDL. Faculty Focus. https://www.facultyfocus.com/articles/equality-inclusion-and-diversity/culturally-responsive-teaching-and-udl/
Dougherty, E. (2012). Assignments matter: Making the connections that help students meet standards. Alexandria, VA: ASCD.
Kieran, L., & Anderson, C. (2019). Connecting universal design for learning with culturally responsive teaching. Education and Urban Society, 51(9), 1202-1216.
Gay, G. (2013). Teaching to and through cultural diversity. Curriculum Inquiry, 43(1), 48-70.
Hedberg, J. G., & Brown, I. (2002). Understanding cross-cultural meaning through visual media. Educational Media International, 39(1), 23-30.
Ladson-Billings, G. (1994). What we can learn from multicultural education research. Educational Leadership, 51(8), 22-26.
Merryfield, M. M. (2000). Why aren’t teachers being prepared to teach for diversity, equity, and global interconnectedness? A study of lived experiences in the making of multicultural and global educators. Teaching and Teacher Education, 16(4), 429-443.
Flores, N., & Rosa, J. (2015). Undoing appropriateness: Raciolinguistic ideologies and language diversity in education. Harvard Educational Review, 85(2), 149-171.
This post is Part I of a two-part series on video selection and use in online courses. Part I provides the reasoning behind understanding course videos selection by course developers as a curatorial process. Part II will explore video curation in practice in course development and provide a course design perspective on video presentation and management issues.
Recent Video Use Trends
In September of 2020 the enterprise video company Kaltura Inc. conducted its seventh annual State of Video in Education 2020 report. The report included responses from across the education system spectrum with higher education institutions making up 53% of all respondents (Figure 1.).
Figure 1. Percentage of respondents to video survey by education sector.
This report described how remote teaching-driven course changes impacted video adoption and use in education. Remote teaching and learning was the most common use of video (83% of respondents). Lecture captured as video was used by 69% of the responding institutions.
The executive summary identified a number of key insights and trends related to changes in video use in education. A select few can be seen below:
Use of video for remote teaching and learning grew by 28% over 2019.
Video use is viewed as positive. Respondents (84%) saw video as having a positive impact on student satisfaction, 73% seeing video increase student achievements and 76% believe it increased instructor satisfaction.
Students as creators of video increased by 13% from 2019 to 2020.
In higher education there was rising video use for remote teaching, lecture capture, and flipping the classroom.
Actual growth in the use of video for remote teaching and learning grew by 28%.
A majority of respondents (68%) want to continue to blend traditional teaching with today’s virtual innovations; such as video.
In some ways this is not surprising. This past year forced many instructors in higher education to convert face-to-face courses to remote instruction. Much of that transition was accomplished with synchronous sessions via ZOOM or some other video conferencing program. Live video conference sessions, if recorded, also served as a support resource for students. In response to the challenges of the past year both live and recorded video were adopted to make remote learning doable. Fully online courses do not have this live element as they are asynchronous and did not have to adapt in this way.
In asynchronous courses at Oregon State University our Ecampus course developers utilize video differently. Video is as a key media element in delivering course content to learners, promoting faculty presence, and to build depth into projects and assignments. Video content may be produced internally by course developers (e.g., instructors) and used in courses via an enterprise video system (e.g., Kaltura). Video content may also be sourced from external video-based social media sites (e.g., YouTube and Vimeo) or educational and commercial collections (e.g., Kanopy or Amazon) and via syndicated video sources (e.g., podcasts and Twitter).
Given the plethora of video available and a trend toward increased video integration into instruction the challenge to course developers is the selecting, managing, and presenting video content to support and compliment course learning outcomes. Ultimately this also becomes a course design challenge for instructional designers who must adapt to manage the integration of increasing levels of video in the course in a way that makes sense from a pedagogical perspective as well as visual design aesthetic.
Course Developers as Media Curators
What is a Curator?
The growing value of video in the experience of a course suggests that course developers (e.g., instructors) consider a new way of thinking about how video is selected, managed and presented. In essence, I am suggesting that for a given course the course developer serves as a curator of video content.
But what is curator? Should a course developer really think like a curator? How might curated media shape course development and instructional design?
In order to explore this notion of course developers as media curators a bit more I would like to share the definition of what a curator is from the American Alliance of Museum (AAM) Curators Committee (2009). The preamble to the curator core competencies of a curator defined the term curator as:
Curators are highly knowledgeable, experienced, or educated
in a disciplinerelevant to the museum’s purpose or mission.
Curators are further described as having nine core competencies and related applied skills. The competencies are:
Collection planning Scholary Research Exhibition Development
Collecting Object Research Education
Collection Care Applied Research Outreach & Advocacy
In Figure 2. we see these same foundational roles expressed by the AAM coupled with a definition of curator and description of the work of a curator.Also included is the domain of the work. Those domains are preservation, research, and communication. The global context of curation is, in this definition, a museum. The more discrete context is the exhibition, or exhibit application. Yet it is all part of a curator’s work.
Figure 2. Definition of the term curator and select context example.
What we see in this definition in Figure 2. is the premise that curators select, gather, care for, and prepare presentations of single items that in aggregate make up a curated collection. That collection becomes a resource and object of education, outreach, advocacy and presentation.
This makes the act of curatorship a scholarly and creative practice that is deeply intentional and based upon the definitional parameters of the organization doing the work.
Course Developers – Curators of Video Collections
Now let us think about what an online course developer is and what they do. At Ecampus course developers collaborate with instructional designers to plan an online course. Instructional designers advise and take content selected by the course developer and build that content into Canvas, our learning management system. The created courses are then shared with students. Course developers are considered content experts much like museum curators are. Let’s look at that a bit more closely.
In Figure 3. below we can see a comparison between the definitional role and duties of a museum curator and course developer. There are striking parallels between these roles. So much so that it would seem reasonable to think about what a course developer does as also a curatorial practice. A practice focused on the learning content, including video, for a given course.
Perhaps the greatest difference between these to two curatorial practices is the context of each. In asynchronous course development it is not uncommon for instructors to perform many of these same functions as museum curators but on a more discrete scale. The scope and context of their focus is obviously different.
In essence a course developer actively gathers and in may cases, creates unique course elements that form the curated media collection for a course. That collection of texts (readings), images, web resources and video is then used for education, research, and perhaps outreach with a constant eye on student access to media. Ultimately a course media collection is intended to permit the course developer to fulfill the purpose of the course and guide students in achieving the course learning outcomes.
The physical design of the course, with its media collection, is the domain of the instructional designer. The collaboration between the course developer and instructional designer are key in preparing the course as an “education exhibition” of sorts that has clear learning outcomes.
Course Video Selection: The Art of Curatorship
We began this discussion with the importance of video in online course development and design. With that in mind it is logical that video curation is an important element of course-wide media collection identification.
Video collection, cataloging, arranging and assembling for display in a course fits quite well within the parameters of curating. Any curation is also about a level of storying, opportunities for engagement, information sharing and perspective sharing (Potter, 2017). In course development these processes as applied to course media, and in particular video, have the potential to create and shape the nature, experience, and associated learning in an online course.
In making decisions about video use in online courses, a course developer would apply their knowledge and expertise to curate the selections. Clear learning outcomes provide a pedagogical and content structure to the video curation process. Once a video collection is established other decisions may come into play that reference an aesthetic for the collection. This is the art of curatorship.
The art of curatorship has been viewed as closely aligned to a design process (Shuey, 2014) and may be guided by an interpretation of the universal visual design principles as conceptual guides to the education exhibition that is the online course. In this sense the curator is not thinking as much about the collection items per se but more about how the collection fits together to provide and support a narrative, flow, or education scaffolding for the course.
Thinking Like A Curator
As an exercise in curatorial thinking let’s take some re-interpreted concepts of visual design and see if they help us think through how we curate not only individual videos but also a video collection. This brief list includes accompanying questions that are informed by the identified principle and may shape the curation of video. In these examples found videos are outside video sources where created videos are those made by the course developer.
Balance: What is the intended balance between: Created and found curated videos? Permanent video and temporary (single-use) video content?
Emphasis: How does found video reinforce or extend created video? Is there a particular focus or intention of video use?
Movement: Is there a scaffolding of curated video that matches the scaffolding of the course progression? How does the video curation contribute to that progression?
Pattern: Is curated content focused, more general in nature, or quite diverse in source, topic or message?
Rhythm: Does video use and viewing support or promote a rhythm of engagement for the course that compliments course objectives?
Repetition: Are curated videos reinforcing similar ideas or concepts? Are videos used consistently for certain aspects of the course (i.e., narrated lectures)?
Proportion: Does the video collection time commitment fit within the time expectations for the course? What is the ideal proportion of video to text, image, and other course media?
Variety: Are curated videos from different content sources and types? What is the ideal balance for the course?
Unity: Does the video collection promote a sense of wholeness to the course? Could the video collection, on its own, communicate identifiable ideas, patterns of ideas, or a range of perspectives on a topic?
Does video accessibility contribute to the overall course accessibility?
In working through this exercise, we begin to move beyond video collecting by subject toward a more complete analysis of video collection selection and use that includes intertwined pedagogic and aesthetic considerations. This helps create a video collection that is intentional in its item selection, organization and use.
Final Thoughts
Recent research by Kaltura Inc. indicates that video use in education is on the rise in the past year. A continued growth of access to video and ability to create video coupled with an interest in integrating video in education efforts suggests course developers have a challenging task regarding media selection and use.
This article presents the idea that course developers, whether obvious or not, are actively engaged in a curatorial process regarding media selection and use. In addition, because of the importance and prevalence of video, its curation is presented as a key element of the larger course media curation effort. Lastly, we have explored how video collections contribute to academic and aesthetic value of a course and provided some key considerations based upon extending classic visual design principles to a curatorial practice.
It is interesting that the term curation has Latin roots in the verb curare; which means to take care of. Course developers conducting intentional video curation contribute to meaningful media curation for a course. This engagement in the practice of a curator is truly a professional act of caring about the quality of course development and the impact on student learning.
In Part II of this series we will address the practice of video curation in the context of an online course and explore instructional design considerations for video use that balance and complement a sample course video collection.
References
American Alliance of Museums. (2009). Curators Committee (CurCom): Curator’s core competencies. https://www.aam-us.org/professional-networks/curators-committee/
Kaltura Inc. (2020) The state of video in education 2020: Insights and trends [seventh edition].
https: //corp.kaltura.com/resources/the-state-of-video-in-education-2020/
Potter, J. (2017). Curation. In K. Peppler (Ed.), The SAGE encyclopedia of out of school learning (pp. 4-6). SAGE Publications Inc., Thousand Oaks, CA.
Shuey, G. (2014, October 21). The art of content curation. RELEVANCE.
https: //www.relevance.com/the-art-of-content-curation/
Wikipedia (n.d.). Definition of term collection.
https://en.wikipedia.org/wiki/Collection_(artwork)
Wikipedia (n.d.). Definition of term curator.
https://en.wikipedia.org/wiki/Curator
Memory plays the central role in learning – it is “the mechanism by which our teaching literally changes students’ minds and brains” (Miller, 2014, p. 88). Thus, understanding how memory works is important for both instructional designers and instructors. According to modern theories, memory involves three major processes: encoding (transforming information into memory representations), storage (the maintenance of these representations for a long time), and retrieval (the process of accessing the stored representations when we need them for some goal or task) (Miller, 2014). Let’s briefly review these processes and see how they may inform our course design and instruction.
Encoding – What Is the Role of Attention and Working Memory?
How does encoding happen? We receive information from our senses (visual, auditory, etc.), and then we perform a preconscious analysis to check whether it is important to survival and if it is related to our current goals. If it is, this information is retained and will be further processed and turned into mental representations. Thus, attention is the major process through which information enters our consciousness (MacKay, 1987). Attention is limited, and it is to some extent under voluntary control, but it can be easily disrupted by strong stimuli. Attention is crucial for memory, and without attention we cannot remember much (Miller, 2014).
How attention is directed depends on the way the content is presented and on the nature of the content itself (Richey et al., 2011). If the content is intrinsically motivating for the student, it will catch their attention more readily. But beyond that, the manner we design our instructional materials can influence how learners focus their attention to select and process the information, and in turn on what and how much gets stored in their memory. For example, we can ensure that students are guided to the most relevant content first by making that content more visually salient. Or we can tell an engaging story to focus their attention to the concepts that come next.
Baddeley and Hitch’s multicomponent model of working memory (1974)
Working memory is a concept introduced in the 1970s by Alan Baddeley. This model describes immediate memory as a system of subcomponents, each of them processing specialized information such as sounds and visual-spatial information. This system also performs operations on this information and are managed by a mechanism called the central executive. The central executive combines the information from the various subcomponents, draws on information stored in long-term memory, and integrates new information with the old one (Baddeley, 1986).
Some researchers consider attention and working memory to be the same thing; while not everyone agrees, it is clear that they are highly interconnected and overlapping processes (Cowan, 2011; Engle, 2002). Attention is the process that decides what information stays in working memory and keeps it available for the current task. It is also involved in coordinating the working memory components and allocating resources based on needs and goals (Miller, 2014).
The capacity of each of the working memory components is limited. However, these components are mostly independent: visual information will interfere with other visual information, but not much with another type such as verbal information (Baddeley, 1986). Therefore, the most effective instructional materials will include a combination of media, such as images and text (or better yet, audio narration), rather than just images or just text. Graphic by Cheese360 at English Wikipedia is licensed under CC BY-SA 3.0
Storage – How Fast Do We Forget?
Ebbinghaus’s forgetting curve (1885)
In the late 1800s, Hermann Ebbinghaus conducted his famous series of experiments on the shape of forgetting. The result was the forgetting curve (also called the retention curve), which is a function showing that the majority of forgetting takes place soon after learning, after which less information will be lost (Ebbinghaus, 1885). A recent review of studies on the retention curve concluded that the rate of forgetting may increase up to seven days, and slows down afterwards (Fisher & Radvansky, 2018). This interval is useful to consider when planning instruction. A well-designed course will include sufficient opportunities for practice and retrieval during this time, so as to minimize the forgetting that naturally occurs.
Graphic from MIT OpenCourseWare is licensed under CC BY-NC-SA 4.0
Retrieval – How Do We Get It Out of Our Heads and Use It?
While long-term memory is considered unlimited, retrieval (or recall) can be challenging. Its success depends on a few factors. To retrieve memory representations, we use cues—information that serves as a starting point. Since a memory can include different sensory aspects, information with rich sensory associations is usually remembered more easily (Miller, 2014). Visual and spatial cues are particularly powerful: memory athletes perform some mind-blowing feats by using a special technique called “the memory palace”—imagining a familiar building or town and placing all content inside it in visual form (to learn more about this technique, check out this TED talk by science writer Joshua Foer).
Recall is also influenced by how the information was first processed: deep processing (focusing on meaning) will yield superior retrieval performance compared to shallow processing (focusing on superficial features like some key words or the layout of the information). However, equally important is a match between the type of processing that happens during encoding and the one that happens during retrieval (Miller, 2014). For instance, if the final exam contains multiple-choice questions, learners will perform better if they also practiced with multiple-choice questions when they learn the content. Finally, emotions have been shown to boost memory (Kensinger, 2009), and even negative emotions (such as fear or anger) can have a strong effect on recall (Porter & Peace, 2007).
Conclusion – Implications for Instruction
What can we do to maximize our students’ memory potential? Based on these memory characteristics, here are a few strategies that can help:
Make use of graphic design and multimedia learning principles to create attention-grabbing, well-organized instructional materials that include a combination of media.
Include plenty of retrieval practice activities, such as polling during lectures, quizzes, or flashcards. The website Retrieval Practice is a fantastic resource for quick tips, detailed guides, and research. Top things to keep in mind:
Boost retrieval practice through spacing (spreading sessions over time) and interleaving (mixing up related topics during a practice session).
Make sure you plan some sessions for the critical seven-day period after introducing the material.
Consider teaching students the memory palace technique for content that requires heavy memorization.
Support every type of content visually where possible.
Encourage deep processing of the material, for example through reflections, problem-solving, or creative activities.
Ensure that students have opportunities to engage with the material during learning in the same way as they will during the exam.
Try to stimulate emotions in relation to the content. While negative affect can help (for example, recounting a sad story to illustrate a concept), it is probably best to focus on positive emotions through exciting news, inspiring anecdotes, and even more “extrinsic” factors such as humor, uplifting music, or attractive visual design.
Using these strategies will help you create learning experiences where students encode, store, and retrieve information efficiently, allowing them to use it effectively in their lives, studies, and work. Do you have any related experience or tips? If so, share in a comment!
References
Baddeley, A. D. (1986). Working memory. Oxford University Press.
Cowan, N. (2011). The focus of attention as observed in visual working memory tasks: Making sense of competing claims. Neuropsychologia, 49(6), 1401–1406. https://doi.org/10.1016/j.neuropsychologia.2011.01.035
Ebbinghaus, H. (1885). Memory: A contribution to experimental psychology.
Engle, R. W. (2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11(1), 19–23. https://doi.org/10.1111/1467-8721.00160
Fisher, J. S., & Radvansky, G. A. (2018). Patterns of forgetting. Journal of Memory and Language, 102, 130–141. https://doi.org/10.1016/j.jml.2018.05.008
Kensinger, E. A. (2009). How emotion affects older adults’ memories for event details. Memory, 17(2), 208–219. https://doi.org/10.1080/09658210802221425
MacKay, D. G. (1987). The organization of perception and action: A theory for language and other cognitive skills. Springer New York. http://dx.doi.org/10.1007/978-1-4612-4754-8
Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press.
Porter, S., & Peace, K. A. (2007). The scars of memory. Psychological Science, 18(5), 435–441. https://doi.org/10.1111/j.1467-9280.2007.01918.x
Richey, R., Klein, J. D., & Tracey, M. W. (2011). The instructional design knowledge base: Theory, research, and practice. Routledge.
This is the paradox of failure in games. It can be stated like this:
We generally avoid failure.
We experience failure when playing games.
We seek out games, although we will experience something that we normally avoid. (Juul, p. 2)
As a continuation from my last blog post considering grades and Self-Determination Theory, I wanted to take a brief side-quest into considering what it means to experience failure. Jesper Juul’s The Art of Failure: An Essay on the Pain of Playing Video Games will provide the main outline and material for this post, while I add what lessons we might learn about feedback and course design in online settings.
Dealing with Failure
Juul outlines how games communicate through feedback using the theory of Learned Helplessness. Specifically, he focuses on Weiner’s attribution theory, which has three dimensions:
Internal vs. External Failure
Internal: The failure is the fault of the player. “I don’t have the skills to defeat this enemy right now.”
External: The failure is the fault of the game. “The camera moved in a way that I couldn’t see or control and resulted in a game over.”
Stable vs. Unstable Failure
Stable: The failure will be consistent. No recognition of experience gained or improvement. “No matter what I do, I can’t get past this challenge.”
Unstable: The failure is temporary. There is a possibility for future success. “I can improve and try again.”
Global vs. Specific Failure
Global: There is a general inability preventing success. “I am not good at playing video games.”
Specific: Poor performance does not reflect on our general abilities or intelligence. “I’m not good at flight simulators, but that doesn’t mean I’m bad at all video games.”
In general, a combination of Internal+Stable+Global failure feedback would contribute most strongly toward a player adopting a learned helplessness mindset. There is a potential parallel here with course design: when a student does not do well on an assessment, what kind of feedback are they receiving? In particular, are they receiving signals that there is no opportunity for improvement (stable failure) and that it shows a general inability at the given task (global failure)? Designing assessments so that setbacks are unstable (offer multiple attempts and a way for students to observe their own improvement over time) and communicating specific skills to improve (make sure feedback pinpoints how a student could improve) would help students bounce back from a “game over” scenario. But what about internal vs. external failure? For Juul, “this marks another return of the paradox of failure: it is only through feeling responsible for failure (which we dislike) that we can feel responsible for escaping failure (which we like)” (p. 54). This importance of internal failure aligns with what we know about metacognition (Berthoff, “Dialectical notebooks and the audit of meaning”) and the numerous benefits of reflection in learning.
Succeeding from Failure
Now that we have an idea on how we deal with failure, let’s consider how we can turn that failure into success! “Games then promise players the possibility of success through three different kinds of fairness or three different paths: skill, chance, and labor” (Juul, p. 74):
Skill: Learning through failure, emphasis on improvement with each attempt. (This is also very motivating by being competence-supportive!)
Chance: We try again to see if we get lucky.
Labor: Incremental progress on small tasks accumulates more abilities and items that persist through time and multiple play sessions. Emphasis here is on incremental growth over time through repetition. (Animal Crossing is a great example.) (This path is also supported by Dweck’s growth mindset.)
Many games reward players for all three of these paths to success. In an online course, allowing flexibility in assignment strategies can help students explore different routes to success. For example, a final project could allow for numerous format types, like a paper, podcast, video tutorial, interactive poster, etc. that students choose strategically based on their own skills and interests. Recognizing improvement will help students with their skills and helping students establish a routine of smaller, simpler tasks that build over an entire course can help them succeed through labor. Chance is an interesting thing to think about in terms of courses, but I like to think of this as it relates to content. Maybe a student “gets lucky” by having a discussion topic align with their final project topic, for example. For the student in that example, that discussion would come easier to them by chance. Diversifying content and assignment types can help different individuals and groups of students feel like they have “lucky” moments in a course.
Reflecting on Failure
Finally, how do games give us the opportunity to reflect on our successes and failures during gameplay? Juul outlines three types of goals that “make failure personal in a different way and integrates a game into our life in its own way” (pp. 86–87):
Completable Goal: Often the result of a linear path and has a definite end.
These can be game- or player-created. (i.e., Game-Driven: Defeat the ghost haunting the castle. Player-Driven: I want to defeat the ghost without using magic.)
Transient Goal: Specific, one-time game sessions with no defined end, but played in rounds. (e.g., winning or losing a single round of Mario Kart.)
Improvement Goal: Completing a personal best score, where a new high score sets a new goal.
For Juul, each of these goal-types have different “existential implications: while working toward a completable goal, we are permanently inscribed with a deficiency, and reaching the goal removes that deficiency, perhaps also removing the desire to play again. On the other hand, we can never make up for failure against a transient goal (since a lost match will always be lost), whereas an improvement goal is a continued process of personal progress” (pp. 86–87). When thinking about your courses, what kinds of goals do you design for? Many courses have single-attempt assignments (transient goal), but what if those were designed to be improvement goals, where students worked toward improving on their previous work in a more iterative way that replaced old scores with new and improved scores (improvement goal)? Are there opportunities for students to create their own challenging completable goals?
I hope this post shines a light on some different ways of thinking about assessment design, feedback types, and making opportunities for students to “fail safely” based on how these designs are achieved in gaming. To sum everything up, “skill, labor, and chance make us feel deficient in different ways when we fail. Transient, improvement, and completable goals distribute our flaws, our failures, and successes in different ways across our lifetimes” (Juul, p. 90).
In 1989, Kimberlé Williams Crenshaw, a lawyer and scholar of Critical Race Theory (CRT), coined the term intersectionality to describe the multiple and layered oppressions experienced by African American women. Over time, this term has been used to describe many aspects of social identity, particularly focusing on race, gender, and class oppression. Intersectionality allows us to consider the impact of multiple oppressions on individuals and groups. For example, asking what it means to be poor in the United States is different from asking what it means to bea poor, Black, woman in the United States, which is different from asking what it means to be a poor, Black, disabled woman in the Southern United States.
Intersectionality matters because, if we don’t recognize and support our most marginalized citizens, they will continue to fall through the cracks. In colleges and universities, this means that our most marginalized students may need additional support to perform to their full potential. Addressing one source of oppression may not provide enough support to students who are working to overcome multiple sources of oppression.
Disability in Higher Ed
Disability is an obstacle for many college students. Consider these statistics:
19% of undergraduate students report having a disability.
28% of American Indian/Alaska Native students reported a disability.
21% of White students reported having a disability (rounded to nearest percent).
17% of the students with disabilities are Black. (National Center for Educational Statistics [NCES], 2019).
When considering disability–or any other identity–we need to consider how other characteristics might compound the marginalization of students with disabilities. Let’s consider how race intersects with disability.
While the percentage of Black people with disabilities in higher education is lower than the percentage of White people with disabilities in higher education, in the general population, the reverse is true. According to Courtney-Long, E.A., Romano, S.D., Carroll, D.D. et al. (2017), 1 in 4 Black people have a disability, while 1 in 5 White people have a disability. This means that more white people with disabilities are accessing and progressing through higher education.
It is also important to recognize that the actual percentages of students with disabilities is higher as many students choose not to disclose their disabilities to their institutions. According to one study, “9% of students who identified as disabled did not disclose this information to their college or university” (Taylor & Shallish, 2019, p. 10).
There are clearly opportunity and equity issues that disproportionately impact students of color with disabilities in higher education.
Yet, when we work to create learning environments that are inclusive of students with disabilities, we often neglect to address intersecting sources of oppression. For example, accessibility requirements do not consider how disability intersects with other oppressions, such as class or race.
Universal Design for Learning
Universal Design for Learning (UDL) is an approach that is commonly cited as a way to meet the needs of all learners. UDL includes a framework with three general principles (multiple means of engagement, multiple means of representation, and multiple means of action and expression) each of which includes multiple guidelines and checkpoints for actual practice (CAST.org, n.d.). The goal of UDL is to increase access and usability for the greatest number of people possible. A UDL approach is structured and practical and, despite the critiques included here, is lauded for its utility by course designers and teachers alike.
UDL, however, does not meet the needs of all learners, particularly our most marginalized learners. Let me repeat: UDL does not meet the needs of our most marginalized learners, as much as we would like to believe it does. Let me highlight a few of the reasons for this.
As Dolmage (2017) explains in the book Academic Ableism: Disability and Higher Education, UDL’s emphasis on universality is problematic because universality is connected to normativity. (p. 134). Dolmage (2017) states that UDL has gained recognition by appealing to the majority, but in doing so “the needs of the majority once again trump the needs of those who have been traditionally excluded—people with disabilities” (p. 135). UDL is viewed as a framework for addressing the needs of disabled students, but its actual emphasis is on meeting the needs of the majority.
With its emphasis on “multiple means” UDL aims to include multiple learner identities and preferences; however, it “overlooks the importance of feedback from its own users” (Dolmage, 2017, p. 126). In this way, UDL ignores the individual circumstances of actual students.
By focusing on the “means,” over the students themselves, UDL is not an intersectional approach to design and teaching. Defining what a Universal Design looks like without considering the particularized realities of actual students results in the continued marginalization and erasure of students who are not in the majority.
UDL has popularized educational practices that serve many students, but in doing so, it has effectively erased the needs of some of the most marginalized students–those with disabilities. Those students with disabilities who are also part of other oppressed groups are increasingly at a disadvantage.
There’s no doubt that UDL is an incredibly useful tool and makes our course designs better, but we must not fail to recognize that UDL is not a panacea. UDL should be one of many tools we use to meet the needs of students, but let’s not forget that we need a truly intersectional approach to design and teaching. Without this, we, unwittingly or not, are contributing to the marginalization and erasure of our most disadvantaged students.
Courtney-Long, E.A., Romano, S.D., Carroll, D.D. et al. (2017). Socioeconomic Factors at the Intersection of Race and Ethnicity Influencing Health Risks for People with Disabilities. J. Racial and Ethnic Health Disparities, 4, 213–222. https://doi.org/10.1007/s40615-016-0220-5
Dolmage, J. (2017). Universal Design. In Academic Ableism: Disability and Higher Education (pp. 115-152). Ann Arbor: University of Michigan Press. Retrieved June 8, 2020, from www.jstor.org/stable/j.ctvr33d50.7
Taylor, A. & Shallish, L. (2019). The logic of bio-meritocracy in the promotion of higher education equity, Disability & Society, DOI: 10.1080/09687599.2019.1613962
U.S. Department of Education, National Center for Education Statistics. (2019). Digest of Education Statistics, 2017 (2018-070), Chapter 3. Retrieved June 10, 2020 from https://nces.ed.gov/fastfacts/display.asp?id=60
