Evaluation

Evaluating an online course can be more complicated than a traditional course because there are often more components that need to be assessed, such as technology, access to student support, and type of content shared. Filling out the Quality Matters Checklist before the course begins can help target potential issues so they can be addressed before the course begins. Most universities require students to fill out end of semester course evaluations. These evaluations can sometimes be useful in adjusting course design and structure for subsequent semesters. However, these summative evaluations are offered too late in the semester to have any impact on the current semester, and often faculty do not have input on the questions asked. Therefore, it can be useful for instructors to provide students with the opportunity to complete  evaluations of the course at various points during the active semester- Canvas allows for the creation of anonymous surveys, so instructors can create short feedback surveys for students to fill out early to mid-semester during the course. These formative evaluations can give instructors the opportunity to discover any serious design flaws or confusing elements in a course, in a timely enough manner to be able to make modifications to the course. Keep in mind that some students may feel uncomfortable filling out an evaluation if it is not anonymous. When designing evaluations, keep them as short as possible to increase the likelihood that students will take the time to fill them out. Also, be sure to clearly and separately evaluate the technology, course content, and instructor. If distinctions are not clear, students might evaluate content poorly when in reality it was the technology that caused problems. Therefore, it is also important to use terminology that the students understand. For example, I have seen evaluations where students were unclear exactly what was meant by “content,” and therefore in the content section they actually evaluated the delivery method. A better term for content might be “course documents” or “course material.” The feedback from formative and summative evaluations can be extremely valuable for instructors in determining which components/aspects of a course assist the learning process and which do not.

Canvas also provides other means of helping you gauge the success of your online course. In addition to preserving all student assignments for later reference, Canvas also provides some basic analytics that can be useful. The “View Course Analytics” page (accessed from a button on the right side of the home page), provides basic summary statistics on: course activity by date, on-time/late/missing assignment submissions, and assignment grade averages. You can also see similar information organized specifically for each student.

Some departments and programs encourage faculty to ask colleagues to evaluate their online courses for constructive feedback. Colleagues can be drawn both from within and outside the department.

Student learning is of course the most important consideration in evaluating an online course, and can also be the most difficult to assess. In addition to reviewing student grades, instructors can also incorporate quizzes (even ungraded) at the beginning of modules and then give the quizzes again at the end of the modules to help judge if student understanding has changed. Another option is to create an optional discussion board at the end of each module called something like “Reflections” or “Moving Forward” where students can post one thing they learned, one thing they had trouble understanding, one thing they found most interesting/surprising, or one thing they related to most personally.

If you are implementing a teaching strategy that is new to students and might need an adjustment period, then it can be particularly important to do a beginning student course evaluation so you can compare it with an end of course evaluation. Sometimes students will initially dislike a teaching approach they are not used to, but once they’ve adjusted, find they learn more and like the format.

Whether for your students or colleagues, have a clear vision of how you want to structure your evaluation. Then, spend time checking to see that your questions and methods of evaluation are the most appropriate means. Also, make sure to allow enough time (and when needed and possible, money) to address any problems that might come to light from the evaluation.