Go Ahead, Ask Me Anything.
In anesthesia training, and maybe in other specialties as well, one of the first things that happens when you start your training is that you take the American Society of Anesthesiology certification boards. I don’t know if this is still true, with computerized testing, but when I was a CA-1, within the first two or three months of the first year of residency I had to sit for the boards. Of course first-year anesthesia residents aren’t taking them for certification purposes. The idea behind testing people who don’t know anything yet is, I think, to establish a baseline which can be built on every year of training, until by the last year the resident can actually answer the questions and pass the exam. I thought this was totally stupid and pointless, a waste of an otherwise beautiful summer Saturday. A recent New York Times article now has me thinking maybe it isn’t as useless as I thought.
According to Benedict Carey (http://www.nytimes.com/2014/09/07/magazine/why-flunking-exams-is-actually-a-good-thing.html?action=click&pgtype=Homepage®ion=CColumn&module=MostEmailed&version=Full&src=me&WT.nav=MostEmailed), such pretesting can improve later performance in subjects that are not previously familiar to the test-taker. Here’s Mr. Carey:
“Across a variety of experiments, psychologists have found that, in some circumstances, wrong answers on a pretest aren’t merely useless guesses. Rather, the attempts themselves change how we think about and store the information contained in the questions. On some kinds of tests, particularly multiple-choice, we benefit from answering incorrectly by, in effect, priming our brain for what’s coming later.”
Instead of being a waste of time, my guesses at the answers to the anesthesia certification exam gave me an idea of what I needed to know, eventually, and how the information might be approached. It also, frankly, gave me information about how to study for the test: do a practice test before studying. The pre-test shows me how much I really know.
Even when the subject matter is familiar testing, done correctly, can not only measure progress but be a tool for learning. Here’s an example of why this might be. I studied violin in college. It was always very easy, in my little practice room with the music there on the stand, to convince myself that I had learned a certain piece and committed it to memory. When the time came to actually “use” the information I had memorized, i.e, to perform the piece, I often had slips of memory. When it counted, I found I didn’t know as much as I thought I did, or didn’t know the piece in sufficient depth. Thus the testing, that is, the performance, taught me what I still needed to learn. Here’s Carey again:
“The problem is that we have misjudged the depth of what we know. We are duped by a misperception of “fluency,” believing that because facts or formulas or arguments are easy to remember right now, they will remain that way tomorrow or the next day. This fluency illusion is so strong that, once we feel we have some topic or assignment down, we assume that further study won’t strengthen our memory of the material. We move on, forgetting that we forget.”
This is a very important point in medical education because the focus is so often on memorization. Cramming the night before for a multiple choice test causes the test to be a measurement of feats of memorization, not true understanding. If I stand in the hallway outside the performance space frantically reviewing the transition passages of a Bach partita, my ability to play the piece from memory has much more to do with immediate recall and much less to do with any deep understanding of the chord structure and harmonic arc of the piece.
Perversely, the value of multiple choice pretesting is somewhat counterbalanced by the relative uselessness of multiple choice testing as a measure of understanding. Here’s what Terry Heick of TeachThought says (http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/01/25/the-real-problem-with-multiple-choice-tests/):
“More than anything else, when a multiple-choice question is given to a student in hopes of measuring how well he or she understands something, it manufactures the illusion of right and wrong, a binary condition that ignores the endlessly fluid nature of information. It alters the tone of learning, shifting it away from a constant process of reconciling old thinking with new data, and toward something of a pitch-and-fetch scenario. One question, four answers, and only one of them is right. Just point to the right answer.”