Creating Quizzes that Support Learning
It takes a lot of work to convince learners that rereading their notes is an inefficient way of studying. As an educator, I have talked with students who are confused after they spend hours studying and still didn’t perform well on the test. So what is the problem with rereading?
Think about a time when you have been a passenger in a car going to a new location. If you had to drive there on your own the next time, would you be able to?
Riding as a passenger is similar to rereading. You are a passive participant. Driving there on your own forces you to recall information. Is it left or right at this intersection? How many sets of lights do I go through? This is known as active recall.
Active recall (or retrieval practice) is important; it can triple long-term retention when compared to rereading1. In order to maximize effectiveness, you need to avoid using notes or other cues while trying to retrieve the information.
So, how can we get learners into the driver’s seat?
ELearning provides an excellent platform for encouraging learners to engage in active recall. “Try it Outs” are regular interactions throughout a course that encourage a learner to recall or even apply the information and skills they are learning. These often take on the appearance of a quiz question: they could be multiple choice or select many, true or false or checkboxes. However, they occur during the program, separate from an end of unit quiz or test. Clients often ask “can we collect the marks from those questions?”
Not Recommended: Report individual results of Try it Outs.
Learners are typically on their own when they are working through an eLearning course. There is no teacher present who might be judging or recording their answers. As a result, learners are more willing to take risks and answer questions that would intimidate them in a classroom environment.
If you track the results of Try it Outs and report them for individual learners, you change them from opportunities to practice recalling information, to obstacles that need to be overcome before completion. Learners focus on their score, rather than the information. For a learner this can lead to decreased motivation, decreased willingness to try to recall on their own (in other words they are more likely to cheat), and decreased confidence when it comes to a final test or applying the knowledge and skills in their job2.
You may want to track data on these questions to see how well a question is performing. This data however, would be used to evaluate the questions, not the learners.
Recommended: Provide feedback.
There has been concern in the past that if learners get practice questions wrong they will remember the wrong answer. This can be true with multiple-choice questions if the learner chooses a “distractor” – an answer that is close to the correct answer but not correct itself. A learner who chooses the distractor and does not learn what the correct answer is may continue to remember the distractor. However, when you provide feedback that includes the correct answer, you are making sure that they can study the correct answer. The result is a minimum of 10% improvement in the number of correct responses on the final test.
Recommended: Make them doable but challenging.
Learners need to feel successful when they complete these Try it Outs. Thinking about and answering questions give learners a sense of progress, which helps keep them motivated. But, it is important that the questions create the right amount of challenge. Too challenging and learners can feel defeated, taking away all feelings of success and development. Too easy and the learner doesn’t actually have to work to recall the information. They may also overestimate their abilities and/or underestimate the difficulty of a final test.3
There is a common misconception that most question types, including multiple choice questions, assess rote memorization. This is not the case. Many question types, including multiple choice questions, can be based on applications or scenarios. These questions ask learners to apply their knowledge in authentic workplace situations. We want them to consider the details of what they have learned, and apply it to a situation that they might experience. This is the difference between “How do you stop a car?” and “An animal just ran onto the highway in front of you. There are cars on either side of you. What do you do? “ In some cases, questions can also require the learner to use critical thinking or critical inquiry skills by selecting the “best” course of action by weighting pros and cons, and using judgment.
Practicing recalling information and answering application-based questions improve the ability of the learner to transfer their learning between different scenarios and situations, which is key to on-the-job performance.
Try it Outs are key to creating a course that supports learning. Reporting individual results of Try it Outs increases pressure and decreases the potential for long-term learning. Learners need to have the opportunity to try answering questions on their own. And there needs to be the option to fail in a safe environment.
1, 2, 3 – Statistics and other data from Roediger, H. L. III & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences 15(1), 20-27.