Why Peer Review?

According to a study by Leadership IQ, 26% of new hires fail because they can’t accept feedback (Murphy, 2015). Most students are trained to study for grades and have seldom been given enough training on how to receive feedback and how to make feedback work for them. By the time they enter workforce, they will have a hard time facing feedback from coworkers and supervisors. As instructional designers and instructors, we can help by training students in peer review skills. In doing so, we are preparing our students to be successful in their future career on the one hand; on the other hand, instructors will spend less time grading peer reviewed submissions because of the improved quality of work submitted. It’s a win-win solution. Nothing could get better than this in teaching, right?

How to Create Peer Review Assignments in Canvas?

There are two types of peer review assignments.

Writing assignments with peer review process where peer review effort is not heavily graded. The focus is on improved writing.
Writing assignments with peer review process where peer review efforts is heavily graded. The focus is on training students in peer reviewing.

If your students lack peer reviewing skills, instructors can provide tutorials on how to provide feedback constructively. And instructors can also set up practice assignments where peer review is graded, for at least one or two assignments so that students are given the proper training and practices they need. Here is a video tutorial on how to provide constructive feedback.

If your students have been trained in peer reviewing, I recommend the type of assignments where peer review is not heavily graded. Peer review can be extra credit points, or a small portion of the grade.

To set up peer review in Canvas for an assignment, Log into Canvas course as an instructor/designer, go to the assignment, click “Edit” button to edit the settings for the assignment.

It will greatly help students if you provide clear directions for how you expect students to conduct peer review.

banner image of BA 347 International Business

For example, in BA 347 Research Writing Assignment, the instructor provided the following directions: “Peer Review Feedback guidelines: As you conduct your peer review, remember to praise, criticize appropriately, and be specific with revision strategies.

Identify and describe three strengths in this draft.
Identify and describe three weaknesses in this draft.
How does this draft meet the requirements of the assignment? If not, what is missing?
What should be revised in this writing? Why?
After reading, I was left wondering….”

To set up details for peer review, first we set up a due date for when the draft writing will be due and enter the date in the assignment “Due” area. Secondly, check “Required Peer Reviews” box to enable peer review, and how to assign peer review (manually or automatically), if automatically, enter a number for how many peer reviewers will be automatically assigned for each submission and enter a date for when the peer reviewers will be assigned in the “Assign Reviews” area. Lastly, enter a date in the “Available from … until” area for a “until” date as the date for when the peer review will be due. And explain to your students what these dates mean if this is the first time you assign peer review assignment in your course.

Peer Review Set up in Canvas — How to set up peer review assignment in Canvas. date 1: Individual draft due date; date 2: Peer Reviewers assigned date; date 3: Peer Review due date and assignment becomes unavailable after this time.

Image 1: Canvas Assignment Peer Review Option Set Up

Grading peer review

Once all of the peer reviews have been submitted, if instructors would like to access the actual comments, assess, or add comments of their own, they can do so by going to the speedgrader function. To grade the original submission, simply enter grade point in the Assessment “Grade _____ out of 10” area.

Image 2: Grading Assignment

To grade peer review effort, the instructor would need to set up a separate assignment and name it something like “Peer Review Grade”. Some instructors attach peer review rubric forms so students can attach the forms in the submission for Peer Review Grades. See a youtube video example and its web instructions.

References:

Murphy, M. (2015). Why New Hires Fail. Retrieved on November 17, 2015, from http://www.leadershipiq.com/blogs/leadershipiq/35354241-why-new-hires-fail-emotional-intelligence-vs-skills

Alisa Cooper’s post “Conducting Peer Review Assignments in Canvas” at http://freshmancomp.com/2013/02/19/conducting-peer-review-assignments-in-canvas/

TEXT(S)

There’s been some discussion recently about students and lectures and attention spans.* As conversation about this turned to how students grapple with long form texts in an online course, I thought it might be useful to gather some ideas on course design and working with texts in online courses.

The initial page of the Peterborough Chronicle, marked secondarily by the librarian of the Laud collection. The manuscript is an autograph of the monastic scribes of Peterborough. The opening sections were likely scribed around 1638 (See document). The section displayed is prior to the First Continuation. — The initial page of the Peterborough Chronicle (annotated monastic text)

Structure your course around the texts not the weeks

Some instructors structure their reading intensive course around the books that are read. A module per book with multiple discussion boards and prompts per book. This subtly shifts the focus from what am I doing this week to what is this book doing. For example, in ENG 210 Literatures of the World: Asia, Jeff Fearnside structured the course around the five books they were reading.

Banner depicting a 3D model of the Blue Mosque in Turkey — Banner for the module and book on Turkey

Other course elements allowed summative reflections and integrative questions to address themes throughout the course.

Shift format away from only reading and writing.

If reading, multiple books you might change how students interact with one or more of the books.

In engaging with a text, students might:

watch a play rather than read it
listen to a poem rather than read it (For example, this recording of The Waste Land – poem begins at 16:10)

In responding to a text, students might:

create a video journal or podcast as they progress through text(s)
tweet (or write within 140 characters) summaries of characters or plot themes (similar to the idea of Tweet your thesis)
build a timeline of the narrative (for example, this timeline of Russian History created with student entries for Betsy Ehler’s RUS 233: 20th Century Russian Culture course)

Check for understanding

I’m wary about the idea of reducing student engagement with a text to quizzes but as a feedback mechanism as part of a process and on the way to richer engagement I think quick polls, quizzes, or surveys have a role. You could have quick short post reading quiz or survey to figure out if the students have followed the reading. This would let you respond before the students get further along and further behind.

Other interpretative tools

Pie Charts (Shakespeare) - Link out to a collection of images on flickr analyzing word counts of characters in the plays — Collection of Images Analyzing Word Counts for Characters in Shakespeare’s Plays

There’s a whole range of interpretive tools Digital Humanities tools and engaging with them is well beyond this blog post. However, many projects have available outputs in some form that instructors or students can draw on as they grapple with texts.

A great starting point to explore digital humanities would be to take a look at this overview & contact Jane Nichol the emerging technologies librarian. And as you think through how to use these tools in your course

One example of the type of output you might find is this collection of visualizations of the word counts in Shakespeare’s plays

Collaborative writing and commenting

There are also other ways to dialogue around a text – especially when it’s a short dense text with lots of debate and discussion around the text. There are examples of creating your thesis on a blog as you write or using github or a federated wiki as collaborative authoring tools. However, for many courses google docs (which is integrated into canvas) offers a fantastic, known, and private tool for a course to create a document.

A manifesto for teaching online

As a open experiment let me invite you to engage with a text. The Digital Education team at the University of Edinburgh recently released their 2015 Manifesto for teaching online. There’s a lot to think about in their manifesto and how it works or doesn’t work in our context. Please join in and add your voices this copy for comment – an annotated ‘manifesto for teaching online’. **

*The statement spun off a conversation triggered by the NYT opinion piece on lectures, (which I can’t mention without also noting this thoughtful response)

**If you want to author a new pathway through the text or add resources beyond what comments allow request an account and I’ll add you to the project.

Half a century has passed since educational psychologist Robert Gagné published his influential book, The Conditions of Learning, but his ideas are relevant to online and hybrid learning even today. He presciently wrote, “The real point to be made is that use of a variety of instructional modes is both feasible and potentially effective. . . . What is needed in each case is thoughtful design and management of the learning environment.”

Gagné popularized the concept of nine “instructional events.” Each represents a step in the teaching and learning process:

Gain learners’ attention
Inform students of learning objective
Stimulate recall of prior knowledge
Present stimulus material (content)
Provide learning guidance
Elicit performance (practice)
Provide feedback
Assess performance
Enhance retention and transfer

Do you regularly use these steps in your courses? To address this question, you might consider what your students experience as they work through a typical weekly module in one of your Canvas course sites:

Are they aware of weekly learning outcomes? If not, state them in a weekly overview that serves as the first page in each Canvas module!

How do you gain their attention? Through video, audio, images, interactivity?

Do you use your students’ prior knowledge as a scaffold for the learning in your course? In fact, do you make it a point to assess what they know at the outset your course?

Beyond simply presenting content, how do you provide substantial learning guidance? Do you foster active learning through online interaction student-to-student, student-to-content and student-to-instructor?

Do you frequently assess performance and provide feedback? Do you use weekly quizzes and/or low-stakes writing or problem-solving exercises?

Ultimately, over the course of a term, have you blazed a trail that fully supports your students’ retention and transfer? If so, bravo!

Additional Resources:

And, for a light-hearted view of teaching and learning, here’s a 3-minute video that will definitely gain your attention and may well enhance your retention of this post: Gagné’s 9 Events Featuring Cats

You send out announcements but do your students actually get them? You might wonder if you are doing something incorrectly or if they just aren’t reading them, but, you’ll be interested to know that Canvas allows notification preferences to be modified by the individual user.

Notification preferance list in Canvas — Click to enlarge

Each user has the ability to alter their notifications from Canvas and choose how, when, and with what frequency they want to be notified of several different activities. They even have the option to receive the notifications via text or a different email address that they might check more frequently! What these individual settings mean is that if they select that they don’t want any notifications at all, they aren’t getting news of announcement postings, posted grades, due date reminders, or discussion board posts.

In order to encourage your students to receive notifications, you might think about sending a start of term email with an example of the notification preferences you would suggest based upon your class and explain to them why these specific notifications will help them as the term goes on. In that same email, you can also direct them to the Canvas Guides with step-by-step instructions on how to set up notifications in Canvas You send out announcements but do your students actually get them? You might wonder if you are doing something incorrectly or if they just aren’t reading them, but, you’ll be interested to know that Canvas allows notification preferences to be modified by the individual user.

Reflecting

How do you help your students reflect on your course and integrate what they’re learning into their subject knowledge and worldview? If you want your students to develop metacognition and self-understanding, or to articulate professional identity or a disciplinary perspective – reflection and reflective practice can help them integrate what they learn in your course into how they think.

Work found at https://www.flickr.com/photos/cogdog/4502048268/ / undefined

The Theory

The role of reflection in personal development and academic practice is widely acknowledged as a part of higher order thinking in general and also particularly in AACU’s VALUE rubric for Integrative Learning and rubric for Foundations and Skills for Lifelong Learning . The question is how we incorporate reflection in course design.

Adding it in

Adding reflection as a self-contained activity can be a great step, but we often add such activities as small items at the end of a course, or – from the student point of view – as an afterthought or the extra bit they need to do after they’re finished. Reflections done this way, though of potential benefit, can often easily lapse into superficial form-filling. A better approach is to build reflection into the course, and to scaffold student engagement with the process. This can be much more effective and changes how a student interacts with the reflective activity.

Work found at https://www.flickr.com/photos/cogdog/4708291454/ / undefined

An example: Current Problems in Sustainable Living (PS 399)

In PS 399 Current Problems in Sustainable Living (in the future to be offered as PS 374) Dr. Erika Wolters set out to engage students with the issue of their personal role in sustainability within the context of huge global political systems. The course description is as follows:

“Exploration of the role of individuals in sustainability practices and policies. Special focus is given to an examination of how individuals can make sustainable lifestyle choices in light of policy regulations, technologies, socio-economic conditions, and cultural values.”

The Final Paper

Dr Wolters had set set up her course with three major papers alongside other activities and assessments. Originally, the reflective activity was contained in the final paper which required

“By the end of week 2, please select three personal behaviors […] that you will try to change in order to live more sustainably. Document your starting point and each step along the way. Your final paper will require you to discuss your step-by-step attempts where you were successful, where you met with unexpected difficulties, or any other surprises along the way. Place your personal sustainability experience into the context of your readings about individual actions and impacts.“

This paper sought to integrate practice, reflection, and critical disciplinary analysis. As Dr. Wolters and I discussed the course design and how to help students engage with this activity in an online environment, we were aware of two pitfalls to avoid: students reaching the end of the course and struggling to remember their experience and students spending all of their final paper recounting their experience rather than critically engaging with it.

The redesign

The solution we came up with was to ask students to create journal entries throughout the course documenting and beginning to reflect on their practice. In the ten-week course, they identified their sustainable practice by week 2 and journaled about it in weeks 4, 6, and 8 before writing their final paper in week 10. The journal could either be in written or video diary format. There were any number of tools that could be used to support the video option, but using Canvas’ integrated tools and video recorder enabled students to do so easily and without the cognitive overhead of learning an external tool.

The journal could have been set up in a Canvas discussion board. This setup would have created a shared experience across the class in which students reflected and shared together. However, because the focus of this course was personal reflection, the journal activity was set up using the assignment tool. The video or text reflection was shared only with the professor. The reason for doing this was to create the opportunity for more personal reflections than the student might have felt comfortable posting in a forum.

Work found at https://www.flickr.com/photos/cogdog/5570581236/ / undefined

Work in progress

The course is still underway but halfway through I was able to catch up with Dr. Wolters to find out how it was going. Her key observations so far relate in large part to the changes developed through the availabilty of video as an option for this journal activity. She reported the following:

“I do think it is helping them think about the course differently. It is great having them undertake behavior/habit changes and reflect on the costs vs. benefits. It is really exciting to see how they are embracing the project!”
“[I]t is definitely helping me connect with the students differently. I really enjoy seeing and hearing them vs. just having the one-dimensional responses of the discussion boards.“
“[The video posts] were so much fun I responded with a video comment and then posted bi-weekly announcements as a video. It was fun! I definitely feel more connected to the students this way.”

Interim Conclusion

Although this activity needs further evaluation, it illustrates a way to engage students with academic reflection through encouraging dialogue early and throughout the course. From the initial feedback it seems clear that from the instructor’s perspective it offers opportunities to connect with students throughout the course and enable them to engage with the topic.

Image Credits

All images by Alan Levine (Flickr user Cogdog), used under a CC- BY licence.

“We do not learn from experience… we learn from reflecting on experience.”

― John Dewey

Reflection can be a powerful addition to any module or course both for instructors and for students. Instructors can inform themselves about student learning and whether their teaching is effective. Students can deepen their learning through reflection. Reflecting both on knowledge gained as well as areas of confusion can be valuable.

What types of reflective activities are of use in an online course? Two of the simplest activities to incorporate in a course are the Muddiest Point activity or the One Minute Paper activity. Both are short activities in which students answer questions after a brief reflection on their learning.

Muddiest Point:

What concept was the “muddiest” to you during this week, that is, which concept was most unclear?

Minute Paper:

What was the most important thing you learned during this week?
What important question remains unanswered?

Reflection questions can be general or can be more specific. An instructor may want general feedback on a module in the course or they may want students to reflection on a specific field experience, collaborative group project, difficult concept, lecture, reading, etc. Reflective questions can be general or specific.

In the online classroom in which there are many active learning opportunities, adding in extra reflection activities to an already busy schedule can seem overwhelming. One solution to effectively create reflection activities online is to use the Graded Survey option within Canvas (under Quizzes). Canvas will automatically give the student full credit for submitting the survey.

Reflection does not have to add significant time to the student’s workload, does not have to add significant time to the faculty workload, and can teach students the value of reflection which can be applied to their own lives and to their workplace.

Why Modular Course Design

The Course Development & Training team at Oregon State University Ecampus promotes modular course design in our online courses. Laura Crowder (2011) defines modular content as “a collection of learning resources developed as a single learning object”. The major benefits of modular course design include:

Saving time in the development and updating of course content
Modular components are easily repurposed across courses
Student learning is improved since the content is presented in smaller chunks

Modular Course Design in Online Education

Modular course design has been highly recommended by various pioneers in online education. Stephen Downes (Downes, 1998) stated in “The Future of Online Learning” that, “…Online courses will be modular. A course – especially from the designer level – will no longer be seen as a single unit, but rather, as a collection of component parts, each of which may be replaced or upgraded as the need arises.” Andrea Henne (Kelly, 2009) recommended that “modular course design benefits online instructors and students.” The Institute-wide Task Force on the Future of MIT Education Final Report (2014) suggests “an action plan that includes the goals to identify any new or existing MITx course that could be produced as modules; produce the “sticky” modules associated with these subjects; define a limited set of standalone (“smooth”) modules and produce these; put in place a well-organized repository of existing and new modules and define guidelines for building and credentialing customized courses.”

Modular Course Design at OSU Ecampus

With Ecampus’s move to Canvas, modular course design is even easier to implement. Our online courses are generally formatted into 11 weeks as 11 modules. We used a modular course design template for creating each week’s learning content, which includes:

weekly overviews
learning objectives
pre-quiz
assigned readings
lectures
resources
appropriate activities such as graded and non-graded assignments
discussions
assessments
post-quiz
wrap-up

Our template is very similar to Henne’s template (Kelly, 2009), which consists of learning objectives, see table 1 for comparison of the two templates.

Table 1. Comparison of Global Public Health – H 333’s course design template and Henne’s course design template.

This weekly modular template, however, should not limit us from organizing learning content into even smaller units within a weekly module. Here is an example of two modular learning content units within one week in Global Public Health – H 333. The highlighted boxes show two modular content units within Week 1.

Image 1. Screenshot of Global Public Health – H 333 online course Week 1 Learning module

Therefore, if you have a course that has heavy content within each week, feel free to break them into smaller learning modules instead of putting them together as a long big piece.

Enjoy designing and teaching online in Canvas.

References:
Crowder, L. (2011). How to develop modular content in 4 easy steps. retrieved from http://www.learninghouse.com/blog/publishing/how-to-develop-modular-content-in-4-easy-steps on July 28, 2015.

Downes, S. (1998). The Future of Online Learning. Online Journal of Distance Learning Administration, Volume I, Number 3, Fall 1998. State University of West Georgia, Distance Education Center.

Kelly, Rob. (2009). A Modular Course Design Benefits Online Instructor and Students. Faculty Focus. September 2009. Retrieved on July 24th, 2015 from http://www.facultyfocus.com/articles/online-education/a-modular-course-design-benefits-online-instructor-and-students/

Institute-wide Task Force on the Future of MIT Education: Final Report, July 28, 2013, pp. 49–50.

In the classroom we often discuss readings and other sources of information. Because students are often accustomed to digital communications in which sources are rarely cited, they can benefit from guidance concerning your expectations regarding citation. The instructor for TCE 512, Psychology of the Adolescent, worked with Ecampus to create an infographic through which she provides such guidance.

This infographic is licensed under a Creative Commons license, so you can feel free to download and post it in your own courses. Also, remember that we enjoy collaborating with Ecampus instructors to create innovative resources, so if you have any interesting ideas we would love to work with you!

Analytics in an Ecampus course can be a valuable source of information about students. It can help the instructor quickly “see” student progress through an online course and communicate with students that may be at risk, or look for patterns of behavior that may help guide future course improvements.

Within Canvas there are several different areas in which to review statistics and data to measure the success and activity of students within your course.

1. Course Analytics

From the Home page of a course in Canvas, click on “View Course Analytics”. Course Analytics will display the overall Activity, Assignment status, and Grades within a course across all of the students.

Click on a student name to focus in more detail on one activity and progress within the course.

2. Last Login

Though an instructor can determine the Last Login date for a student from the Canvas Analytics page, the People tool also allows an instructor to quickly check last activity of students within the course.

3. Detailed Page Views and Participation

The Access Report can also be found under the People tool. It allows an instructor to view detailed student activity such as the number of times each page/tool in the course was accessed by a student and the most recent access date.

Currently students do not have access to view their own Analytics but hopefully in the future, they also can see how they are participating in the course to ensure they progress steadily towards success!

Resources

Want to find out more about Canvas Analytics? Review the Help guides at Canvas:

With the migration to Canvas comes many new features and methods for facilitating your course. stock-photo-female-tourist-holding-a-map-890139 The Canvas Guides provide a lot of information, but you may be wondering, where do I even start? Here at Ecampus, we’ve put together a few guides to help you become familiar with some of the tools in Canvas.

First, if you’re wondering, “I did this in Blackboard, but I can’t find it in Canvas; how do I…?”, we’ve created a few design options for that. These design options explore how to adapt features that you’ve used in Blackboard to the new Canvas environment.

We’ve also created some more in depth quick references that help explain how to use some of the most popular Canvas features.

The Quick Reference guides and other helpful Canvas-specific information can be found on our Canvas Faculty Resources page. We also have a list of resources for teaching an online course on our Teaching Resources page where you can find our favorite presentation, web-conferencing, and other tools.

Are there other features you’ve discovered or some you’d like to know more about? Leave your feedback in the comments!

Ecampus Course Development and Training

Providing inspiration for your online class

Peer Review: Why and How

Working with Texts in Online Courses

TEXT(S)

Structure your course around the texts not the weeks

Shift format away from only reading and writing.

Check for understanding

Other interpretative tools

Collaborative writing and commenting

A manifesto for teaching online

Gagné’s Nine Instructional Events Online

Canvas Notification Preferences

Building reflection into your course design.

Reflecting

The Theory

Adding it in

An example: Current Problems in Sustainable Living (PS 399)

The Final Paper

The redesign

Work in progress

Interim Conclusion

Image Credits

Reflecting on Learning

Modular Course Design at OSU Ecampus

Why Modular Course Design

Modular Course Design in Online Education

Modular Course Design at OSU Ecampus

Citations in Online Discussions

Canvas – Analytics

1. Course Analytics

2. Last Login

3. Detailed Page Views and Participation

Resources

Canvas Resources and Guides

Contact Info