Do distractors create false learning in multiple choice questions?

Question

I use quizzes in teaching, but have started to worry that they do more harm than good.

Here's why: We have all learned from "the testing effect" that the act of taking a test is part of the practice that leads to learning.

Say I give my students a multiple choice quiz, immediately after a lecture, to practice. And let's say question looks like this:

"Who wrote The Blank Slate?

[] Steven Pinker

[] Jessie Pinkman

[] Mr Pink"

A student that did not pay much attention at class, may vaguely recognise the second name, so he checks the middle box and moves on.

Has the student now reinforced the incorrect option, and thus created a net negative learning?

And then, as a part two of the same question:

Does it make a difference if the test is done on the same day as the lecture, when the new knowledge is still fluent, or after sleep, when the new knowledge is consolidated (and weak memories culled)?

Part three: Does it make a difference if the quiz is on paper, and thus gives no direct feedback, or online, and can indicate the correct answer?

I have searched Google Scholar extensively, but not found anything on this.

Answer 1

To answer your initial statement: no, I don't think they do more harm than good. The testing effect has been shown overall to improve scores compared to traditional studying (though only for long term memory, see (Roediger & Karpicke, 2006) for an interesting discussion), implying that it is doing more good than harm. This study assumes an overall positive effect without feedback to learners on whether they were right or wrong, and where students who incorrectly retrieved do not "learn" or maintain memory (Kornell, Bjork, & Garcia, 2011). Providing feedback to learners on correct and incorrect responses, however, will increase retention even for students who did not correctly retrieve, and may be one of the best ways for these students to learn (Kornell, Hays, & Bjork, 2009).

Part 1: So far as negative learning from selecting an answer, this directly relates to your third question (as mentioned above). Students who got answers correct (successful retrieval) are hypothesized to better maintain memory, while students who didn't will not better maintain (Kornell et al., 2011). Though this seems obvious, remember that retrieval rates naturally decrease over time (the goal is to maintain a high retrieval rate).

Part 2: In a word, yes. Given this, they each measure different things (and should be used for different purposes). Personally, I use a "post-lecture" quiz as an "exit-ticket" (sorry, background with high school freshman) to quickly check where misconceptions lie. These quizzes are reviewed prior to students leaving the class in order to identify and address misconceptions (formative assessment). I've found that waiting can allow more connections to be made (and thus, higher level responses), but if the material is not encoded correctly false connections can be made (reinforcing incorrect information).

Part 3: Not reviewing the quiz is hypothesized to create a "bifurcation" where those who got answers right better remember, and those who did not get the answers right steadily decrease (again, a natural pattern in retrieval over time). The "testing effect" exists because overall students are theorized to do better, but again, there will be a split in student growth between those who remembered and those who didn't (Kornell et al., 2011). Reviewing the quiz immediately (if paper, go over the answers right after the quiz!) is hypothesized to help all students, regardless of whether they initially got the correct answer or not, and thus would be the recommended method to improve students' knowledge retention (Kornell et al., 2009).

To summarize: review after taking quizzes acts much like a "studying" effect (where those who got right and wrong answers can learn), while quizzes without review are hypothesized to split learning (those who correctly recall retain, and those who fail to correctly recall don't retain). Both groups are able to be helped through quizzing and reviewing answers.

References

Kornell, N., Bjork, R. A., & Garcia, M. A. (2011). Why tests appear to prevent forgetting: A distribution-based bifurcation model. Journal of Memory and Language, 65(2), 85–97. doi:10.1016/j.jml.2011.04.002

Kornell, N., Hays, M. J., & Bjork, R. A. (2009). Unsuccessful retrieval attempts enhance subsequent learning. Journal of Experimental Psychology. Learning, Memory, and Cognition, 35(4), 989–998. doi:10.1037/a0015729

Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255. doi:10.1111/j.1467-9280.2006.01693.x