Medical trainees are struggling with the apparent dichotomy of being able to provide optimal clinical care for their patients as clinicians, yet allowing time to master new knowledge and prepare for high-stakes examinations. Given these time pressures, study time of various educational materials must be “worth it,” by maximizing both quality and retention. The neuroscience of learning has allowed us to start considering in what way we can use the principles of how humans learn to design and test the optimal — and most effective — teaching methods to ensure that students learn. Books such as Make It Stick have acted to popularize this view, and the concepts of interleaving, spacing, and the testing effect are steadily becoming part of the vocabulary of the mainstream medical education community.
Although these concepts are useful in theory, they do not directly prescribe how to practically use them in the medical education setting. Therefore, here are some tips organized around commonly asked questions regarding the application of the neuroscience of learning to the preparation for high-stakes examinations.
Tip 1 – What is the optimal number of questions per topic to leverage the testing effect?
The testing effect or test-enhanced learning is the theory that retrieving information from memory by being given tests causes increased retention, as compared to merely reading, or re-reading, information.1 Interestingly, much of the current literature about the testing effect is from a test designers’ perspective as opposed to a learner’s perspective. For individual learners, maximal leveraging of the testing effect requires effortful retrieval, for example trying short-answer questions rather than merely recall-type questions or recognition-type questions, such as multiple-choice questions.2 Completion of groups of MCQs as study before or after tests is likely beneficial to highlight knowledge gaps. Furthermore, allowing learners to get feedback on their answers is key, and tests should be given routinely to learners, with concepts tested at spaced intervals.3 Unfortunately, the exact number of questions and spacing interval required for knowledge mastery and retention likely varies for each individual learner.
Tip 2 – What is the best frequency of number of retrievals to enhance retrieval practice?
Retrieval practice is very much linked to test-enhanced learning and refers to the practice of retrieving information from memory frequently, as a way of enhancing retention. In addition, spaced retrieval practice (retrieving information after longer intervals) has been shown to further enhance retention.4 In fact, retrieval practice has been shown to be critical to long-term retention, relative to repeated studying.1 Indeed, retrieval practice can be useful even in the absence of feedback, but coupled with feedback, retrieval practice enables learners to develop knowledge that is long-lasting, flexible, and transferable between different contexts.1 However, how often should one be retrieving knowledge in order for these powerful effects to ensue? Unfortunately, the answer is that no one really knows the exact answer, and there is no literature that clearly elucidates it.
In a 2014 interview with Digital Promise, Henry Roediger summarized the concept well:
The real answer is that these answers are unknown and probably differ for different materials and different people. However, we should retrieve more than once or twice — maybe 5-7 times — and at long intervals, say over a period of days. You should practice retrieval more frequently for difficult material and for material you think you are barely retrieving. Try it. It works.4
Again, there is not an exact answer to this question, but, in general, we should aim to retrieve as often as we can, especially with the content that we are less familiar with.
Tip 3 – What is the optimal sequence of learning topics?
In Make It Stick, a number of colorful examples were provided of how shifting topics and returning to them again frequently (interleaving), was a powerful way to enhance learning, rather than just studying a full topic (blocking), before moving on to the next. More recent literature appears to consolidate this idea: In 2016, Sana et al. showed that interleaving topics rather than blocking them had a phenomenal effect on inductive learning of cognitive concepts.5 Thus, perhaps counter-intuitively, when reviewing a course or curriculum, varying the topics and returning to them frequently is beneficial, rather than doing a full topic and then moving onto the next. Allowing the learners to be reminded of the topics or retrieving topics between each other has been shown to beneficial to long-term learning and understanding. Frequent interleaving appears to be the recipe for success.
* * *
All of the tips described above require some introspection on the part of the learner to realize their own strengths and weaknesses. Unfortunately, the average learner often has difficulty with self-assessment.
Learning products like NEJM Knowledge+ that feature adaptive-learning technologies are designed to help learners realize where to concentrate their time to ensure that their learning is efficient and effective.
The neuroscience of learning is still a very nascent topic, and the issues of test-enhanced learning, retrieval practice, and optimal sequencing of topics are just a few of numerous concepts that have been shown to improve long-term retention of knowledge. Further studies examining the specific application of these neuroscience concepts in medical education are needed and will most certainly appear in the near future. At the very least, these tools represent the interesting prospect of allowing these neuroscientific principles to be automatically applied to education in a manner that is practical, cost-effective, and congruent with the time constraints of a busy clinician.
- Roediger HL and Butler AC. The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences 2011 Jan; 15(1): 20-27. >View abstract
- Larsen, DP et al. Test-enhanced learning in medical education. Medical Education 2008 Oct; 42(10): 959-966. >View abstract
- Soderstrom NC and Bjork RA. Learning versus performance: an integrative review. Perspectives on Psychological Science 2015 Mar; 10(2): 176-199. >View abstract
- Francisco A. Ask the cognitive scientist: retrieval practice. Digital Promise. October 15, 2014. >Read full text
- Sana F et al. Study sequence matters for the inductive learning of cognitive concepts. Journal of Educational Psychology 2017 Jan; 109(1):84-98. >View abstract
Here’s how I study. I take copious handwritten notes. For me, knowledge which flows through my fingertips stays better than knowledge which I only see and hear, and long-hand writing with multiple abbreviations works better than typing. At the end of each day’s lectures I make out a test for every lecture I’ve heard. The next day, after making out that day’s tests, I take the test from the previous day. Any question I miss I mark with a star. The next day, after the above steps, I look at the previous starred questions, and put another star behind a question I miss again. I repeat the pattern until 1 – 2 days before an examination, at which point I review all the tests I’ve made out on the information to be covered on the upcoming examination. The system requires a lot of effort but it works well for me. I find it interesting that this jives well with multiple retrieval theory.
In addition, when I make out tests, I ask the same question in different ways, spacing the questions apart. For a non-medical example, on one of my pre-tests a question might be “In what year was the Louisiana Purchase completed?” Further down the page I might write “What important event happened in 1803?” Coming at it form different directions causes me to remember it better. And again, having the knowledge flow through my fingers means I’ll retain it longer than simply reading it or listening to someone say it.
I have been teaching until now to medical students and post-graduates Drs. for many years.Now after 30 years of teaching, I left few years ago students.
The problem to organize the program to achieve finally the goal= to handle totally all most important issues of the theme studied.
The problem or problems are so great and quite different:The organization to find the right method, depends of multiple variables:1)Previous knowledge and experience of pre-Doc.2)Amount of books and Journals to review.3)The time left to study , after finishing his/her work at the hospital or Clinic..4)Important to organize a Program , given the right time needed to each chapter 5)One way , it is to handle a Gant Chart.To have respect the time to do anothers activities, like sleep-eating and resting.
But , finally to do that it is so personal:to study has a personal characteristics, depends too much of how the student is facing the Tests and Exams.Only I can add , that a best way to obtain better better efficiency; it is to try always to have drawings of all chapter and themes, , doing it as well as he can:He can have a good feed-back, if , when to finish each chapter try to draw(memory) :He will be using _=ears-eyes and hands