Internal medicine physicians looking for an alternative to the high-stakes, every-10-years board exam that was required to keep up certification are now able to opt-in to a 2-year Knowledge Check-In (KCI), the longitudinal assessment option from the American Board of Internal Medicine (ABIM) Maintenance of Certification (MOC) program. In 2018, about 7,500 internists and nephrologists tried KCI for the first time, according to ABIM.

“KCIs take about three hours, and include access to UpToDate® during the exam without the need for a personal subscription,” according to ABIM. “Physicians choosing the KCI can take it in either a test center or online, such as from their home or workplace. If taking it in a test center the experience is similar to the traditional 10-year MOC exam, with the main difference being the shorter testing format.”

We spoke with a physician who used NEJM Knowledge+ to prepare for an ABIM Knowledge Check-In, Dr. Patrick Burke, a double-boarded internal medicine-pediatrician, treating medically complex pediatric and young adult patients at Valley Children’s Healthcare, in Central California. He has presented on various topics in his field, more recently focusing on challenges of delivering high-quality, affordable health care to adolescent and young adult patients.

Q: What went into your decision to take KCI instead of the 10-year MOC exam? Can you compare your experiences with each?

A: First, I believe firmly in the educational theory that 10-year exams are insufficient to assess continuing medical education. Second, the traditional exams have felt more exhausting than helpful.

Q: The 2-year assessments might reduce some of the immediate stakes for physicians in that it’s not a one-time fail, but still, you have to pass the same amount of content and do well in the end, and you still need MOC Part 2 points to maintain your certification. How are you thinking about MOC now that you’ve taken a Knowledge Check-In — does it change your approach to MOC?

A: Continual assessments mean continual preparation — and that is an entirely new method of study that does not match past methods of “intensive review courses.”  I am still very much in the process of adapting.

Even in its current form, the KCI serves as an imperfect snapshot that I (A) had enough breadth & depth of knowledge (because I passed) and (B) gave me a few more focus areas in its content feedback.

Q: What are the consequences of doing poorly on the ABIM Knowledge Check-In, and were you worried about taking it again in 2 years?

A: The Check-In bills itself as lower stakes — a single failure does not result in loss of certification and physicians can try again in the next 2-year offering as long as their certification does not run out before then. However, the shorter 3-hour exam covers the entire ABIM blueprint.

I had been focused on the pediatrics side of our practice the last few years, so I knew I needed much more preparation to update and refresh my internal medicine fund of knowledge.

Q: Can you talk about the access to UpToDate? How did that change your experience of taking an “exam”?

A: UpToDate is like having a quick reference. Having used it now in both ABIM’s as well as the American Board of Pediatrics’ pilot exam (called “MOCA peds”), it — in no way — “gives the answer away.” I recall it helped jog my memory once, maybe twice.

Q: One of the main purposes of longitudinal assessment is to emphasize lifelong learning over cramming. With its adaptive capabilities, NEJM Knowledge+ tries to drive long-term retention. How did  NEJM Knowledge+ help you prepare?

A: First and foremost, NEJM Knowledge+ helped me focus.  The [ABIM] Blueprint is too broad to go “an inch deep and a mile wide” in terms of studying. Second, it reminded me that ABIM in particular still asks exceptionally detailed questions about subspecialty practice. Third, the experiences and practice exam helped psychologically set my mind at ease — and I scored as well on the bell curve as I’d suspected. After taking the practice exams, I was able to home in on focus areas based upon my performance.

I appreciated the automatic CME / MOC feature — very helpful given the number of stakeholders and regulators looking over our continuing education.

I also appreciated its “small-business feel,” inviting question “Challenges” and providing personalized responses. The content is not stale — it’s well-tended.

The interfaces also worked smoothly — this is huge in the digital age when a sloppy user interface can sabotage the brief periods we have to learn.

And their references provided me opportunities to focus my reading.

Q: What else did you do to prepare for the KCI? How will you prepare in the future, if at all?

A: I did need to look up additional resources for topic areas that still felt rusty. These were primarily narrative reviews or maybe UpToDate articles.

Q: According to ABIM, you have the option to receive your exam results immediately, and you are also provided, within a few weeks, a complete score report highlighting how you performed in different areas of the exam. Did you review your results right after you took the KCI? How did you feel about how you performed, and will you use the information to decide what to review between Check-Ins?

A: At the time I took the exam, they were still having problems with exam results not displaying afterwards as happened with me.  I actually didn’t get notified of my results and had to check the website myself to find my results letter.

Q: ABIM says that initially, Knowledge Check-Ins will cover the same topics as the traditional exam, but then they want to move toward assessing, “a subset of knowledge relevant to practice in the future.” What is your opinion of the topics covered on the Check-In?

A: I believe focusing the exam is absolutely critical. Pediatrics has already taken that route and the result has been a much more focused review experience. Otherwise, we risk under-studying areas as we try to “cover our bases” to do “good enough” to pass.

Q: What are your thoughts on how the KCI could be improved?

A: As other Diplomates have stated, ABIM has a long way to go in terms of exam performance, technical issues, and exam design. For example, they should explore seriously their “security needs” that require Pearson to administer a web-cam supervised exam experience that feels clunky and outdated. Other Boards do not seem to be so preoccupied with fraud and abuse.