Many physicians are using internal medicine question banks to prepare for their board exams – and the vast majority find these banks to be highly effective, according to a recent survey conducted by NEJM Knowledge+ of more than 900 Physician’s First Watch readers.

Among all survey respondents, 52% used question banks online and 40% used them in print. Textbooks were the only tool used more widely, by 56% of respondents. But internal medicine question banks were considered the most effective tool. Approximately 70% of question bank users ranked them “highly effective,” with a slight edge for online (versus print) question banks.

Internal Medicine Board Review Course Study Methods Effectiveness NEJM Knowledge+

Among young physicians, online question banks were the most popular tools of all, used by 75% of responding residents and fellows.

Curious about how this diverse group of physicians used question banks to prepare for boards in internal medicine and other specialties, we interviewed several respondents to learn more about what they liked most, or least, about question banks and to find out about their study strategies and insights.

Internal Medicine Question Banks “Make You Think”

For Honor Schoech, MD, a Colorado internist who has recertified twice, “Questions are the only way to go. It’s much more effective [than other means]. Answering the questions makes you think about the topic.” Because answering questions requires her to apply knowledge, she finds she retains this knowledge better than more passive learning through reading a textbook or listening to a lecture. Education experts would agree: Active engagement with content through an internal medicine question bank is considered “active learning,” which is known to improve retention. This is an established concept in cognitive science known as “the testing effect” or “test-enhanced learning” that also has been studied specifically in medical education.

Like 75% of our survey respondents, Dr. Schoech used more than one tool to prepare for her board exam. An internal medicine question bank was her dominant resource for studying, but when she wanted to dig deeper into a topic, she turned to textbooks and online evidence-based tools. Her opinion is that there’s too much information to tackle — reading thousands of pages isn’t an effective option. Instead, she says, “The questions target your learning.”

Internal Medicine Question Banks Alert You to Knowledge Gaps

“Question banks are good for finding out what you don’t know,” was a common response of both first-time certifiers and recertifiers when asked the most beneficial result of using an internal medicine question bank. Going through questions revealed knowledge gaps, they said, providing a useful guide to which areas would need more study and review.

Stephen Russell, MD, studied for his 2013 ABIM internal medicine recertification exam by reading a study guide, working through questions embedded in it, and using flash cards. “Prospectively, I thought that studying the ‘core curriculum’ was more what I needed. Retrospectively, I needed the questions more,” reflects Russell, an associate professor of internal medicine and pediatrics at the University of Alabama. For his pediatric boards a few months later, he prepared primarily with a question bank. In both instances, he said, answering questions “both honed my focus on knowledge gaps and allowed for quick learning points in reviewing the answers.” His knowledge base prior to studying was very strong, he figured, considering his daily primary care work in internal medicine and pediatrics — and his regular hospital teaching rounds for both children and adults. Even so, the board review questions were useful in unexpected ways, he says: “The questions did force me to focus on low-prevalence, highly morbid conditions that I may not have routinely seen.”

“A question bank was useful for finding specific gaps where my knowledge was weak,” says another physician who prefers to be called by his initials, N.E., and who is currently a fellow in pediatric intensive care and passed his pediatrics boards in 2012. “You have to do enough [questions] to make it worthwhile, but it is good to find those gaps.”

When he found a gap, he looked up the topic in a textbook. This method of using board review questions as a springboard for sharpening knowledge in a specific area is common. And it explains, perhaps, the persistent popularity of internal medicine books for board review preparation.

Hybrid Methods Work for Many Physicians

Test-taking veteran Liz Gabay, MD, has created her own time-tested combination of tools to successfully certify and recertify in internal medicine, geriatrics, and infectious disease. She typically uses a question bank, textbook, and her own note-taking system. For her most recent round of exams, she prepared by using a question bank on her iPad. If she felt uncertain about a topic or wanted more depth, she turned to textbooks. She continued until test day with her own hybrid study system:

“As I do online questions and read the textbook, if I come across something I should know but don’t, I write it down in a notebook. Starting 6 to 12 months beforehand, I begin reviewing specific content areas. About 2 months beforehand, I ramp it up to 5 hours a week and then up to 10 hours a week right before the exams. As the test gets closer, I go back through the notebook, and if something is still not familiar, I circle it in pencil. When I go through it a second time, if it is still not familiar, I circle in blue. The next time, it is still is not familiar, I use a red pen. The night before the exam, I study only the red circled items one last time.”

This process intrigued us because the concept is similar to what NEJM Knowledge+ does. The adaptive learning platform of our online internal medicine question bank automatically adapts the content for each user, serving up questions based on knowledge gaps indicated by the user’s performance as he/she works through questions. Education experts have observed that most people, when tasked with studying, spend more time than necessary on material they already know, rather than building knowledge in areas they find challenging. Dr. Gabay’s thorough method kept her focused on what she needed to learn.

Some Question Banks Lack Adequate Depth

Choice of question bank may make a difference in how well it works for you as a study tool. Some respondents found insufficient depth in particular question banks, particularly older ones. One user of an internal medicine question bank found it “did not provide much high-yield education of what I most needed to know.”

Respondents to our original survey named two top reasons for choosing particular study tools: “convenience” and “quality of the content/reputation of the provider.” Among users of online questions banks, though, “recommended by peers” was the top factor noted.

A number of physicians we talked with said that for their initial certification, they chose a study tool based on recommendations of instructors and trainees who were one year ahead of them. For recertification years later, many returned to familiar tools they liked, or they tried others that incorporated new technologies.

Knowing Your Own Learning Style May Determine Which Tools You Choose

Self-knowledge about what works best for your own learning style determines some physicians’ choices — and how well that tool works for you. Board review courses came in just behind question banks for “highly effective” ratings in our survey. But Dr. Schoech, who enjoys the camaraderie of educational sessions at conferences, finds that lecture courses do not suit her style as a “visual learner.” She prefers to learn through reading, and an internal medicine question bank meets that need.

As for whether print or online question banks are best, many survey respondents used both, although online questions banks were used more frequently among recent test takers. Of online question bank users, 41% also used print question banks, and of print question bank users, 54% also used online.

Keeping Pace for the Exam with Timed, Online Question Banks

One advantage of online questions banks, respondents told us, is that they have simulated exam settings, which acclimate you to the pace and format of the exam. Although Dr. Russell passed both his internal medicine and pediatrics boards in 2013, he has one regret that he will address in his preparation the next time: “More time spent [practicing] on the computer-based format of questions would have made the recertification experience smoother.”

If you have used internal medicine question banks to prepare for boards, what features did you like most or least? Did you use a question bank alone, or in combination with other review methods? Share your experiences here with other physicians.