The American Board of Internal Medicine (ABIM) released a report [PDF] yesterday from its Assessment 2020 Task Force that recommends replacing the 10-year MOC exam with “meaningful, less burdensome” assessments that physicians could take at home or at the workplace.
In addition, the panel recommends new assessments on cognitive and technical skills, as well as recognizing specialization in practice.
Replacing the ABIM Board Exam for MOC
The new assessments would be a combination of open-book and closed-book tests. The latter would require physicians to memorize medical knowledge that most physicians should know without outside references — and the ABIM would seek consensus from the physician community in advance of such a test to determine what should be on it.
The goal of these smaller, more frequent, lower-stakes assessments would be to provide insight into performance, with the results accumulating over time and culminating in a high-stakes pass/fail decision. The task force recommends that if physicians fail multiple assessments over time, they may ultimately need to take a longer exam or another form of assessment in order to maintain certification.
Likely Removal of ABIM MOC Self-Assessment (Part 2)
Notably, the introduction of more frequent open-book assessments would be redundant with MOC Part 2 self-assessment programs. The task force acknowledges this in the report, stating that the “existing self-assessment component of MOC would likely no longer be a separate requirement.”
This proposal represents a big change from previous years, given that the current MOC process requires that physicians earn 100 MOC part 2 self-assessment points every 5 years and submit at least one MOC activity every 2 years. Just recently, the ABIM announced an initiative with the ACCME to dramatically increase the availability of part 2 self-assessment activities.
Focus on Cognitive and Technical Skills
The task force collected feedback from the community and did research on learning best practices, and on the basis of their findings, made a recommendation to strive to assess the cognitive and technical skills that the public expects from physicians. The report’s authors write that “Technical skills are likely to degrade over time since they are not entirely related to the number of procedures performed, but are more about the dexterity and actual physical skills involved in performing them” and “setting standards would ensure that physicians are educated in procedures that they routinely do.” The ABIM would need to explore new assessment methods, such as simulations, in order to test these technical skills.
The panel acknowledges that some attributes of being a competent physician are hard to test, such as empathy, communication, and teamwork, and recommends removing them from MOC assessments — at least for now. As new methods emerge for testing these skills, such as simulation and hands-on procedure observation, the ABIM would reevaluate the need to test physicians on these types of tasks.
Recognize Specialization
Many specialists are further subspecialized, and the ABIM Assessment 2020 task force recommends recognizing this reality. An endocrinologist, for example, may be so subspecialized that he treats only thyroid disease — should the cognitive and technical assessments that he takes require him to restudy the wider field of general endocrinology just to pass the test? The panel writes, “ABIM will need to consider feasible approaches to recognize these focused areas of practice. In doing so, a strategy for how to represent the scope of practice to the public would be essential.”
Enhance the Utility of Multiple-Choice Questions
The ABIM’s task force found that multiple-choice questions serve their purpose but, as currently written, cannot adequately test many complex decision making processes and hands-on skills.
The report explains:
While multiple-choice questions (MCQs) remain an excellent method of assessing cognitive expertise, advances in technology allow for processing different methods of testing including simulations which mimic real-world practice more closely and are used across disciplines to assess, for example, airline and submarine pilots. Natural language processing and automated scoring techniques offer scalable approaches to measuring the clinical reasoning process and not simply whether a physician is able to pick out the correct answer from a list of options. It is likely that the future of assessment will rely more on these multifaceted approaches, especially when evaluating complex skill sets such as communication or clinical reasoning.
Even simple enhancements to typical MCQs may better assess higher order skills like clinical reasoning. It is possible to focus questions on common misconceptions and undesirable actions. New approaches could allow for open-book exams (a form of which is used on the Certified Public Accountant exam) to acknowledge the abundance of information that cannot be memorized. Different item types including short-answer responses or compact mini-performance tasks could be used to assess higher order skills.
ABIM Assessment 2020’s Recommendations in Context
Although the Assessment 2020 task force has been at work since 2013, this report comes on the heels of several recent changes as the ABIM has responded to controversy and sought greater transparency, a less burdensome process, and clearer relevance to practice. These changes include
- suspending quality improvement and patient safety requirements
- no longer requiring underlying certifications for MOC
- a new partnership with the Accreditation Council for Continuing Medical Education (ACCME) to accept more forms of continuing medical education (CME) for MOC credit
- a more-detailed blueprint and score report for the internal medicine recertification exam
Other ABMS boards are also looking at new approaches to MOC that might improve our understanding on how people learn and retain knowledge.
The ABIM Council and Board of Directors state that they are working on considering these recommendations and getting feedback on them; the ABIM writes that it will keep the public updated on their blog.
What do you make of these recommendations? What do you think the ABIM and other boards should change about MOC examinations?
More from the Learning+ Blog on the ABIM’s changes to MOC:
ABIM Teams Up with ACCME, Allows You to Use CME for MOC
ABIM Practice Improvement Modules — What Next?
ABIM Announces New Blueprint and Score Report for Recertification Exam
ABIM Changes MOC Requirements
Too little too late…done with supporting a corrupt, irrelevant organization…will be recertifying with the NBPS.
I feel it may be going in the right direction. I feel we had gotten away from the purpose of the recertification which was knowledge based which does not necessarily translate to examination.
I am approaching my 3rd certification examination since I completed residency and am nearing the end of my career. I would like to maintain certification beyond but our practice changes over the years and there are just many conditions that you no longer encounter as you might have when you were training in a large residency program 30 years ago.
Somehow making the examinations more relevant to the physician’s actual practice instead of a one size fits all approach would be more appropriate.
Should there be different levels of certification, currently the examination is so high stakes with insurance companies no longer allowing participation, is that really what we wanted???
With a little thought I believe this could be accomplished. If you never see pregnant patients why be tested on medical complications of pregnancy.
I enjoyed the NEJM knowledge plus learning style, thought the Pearlman MOC modules were much more specifically directed at a specialist level-Boston Prep Score for colonoscopy as an example. I attended two national conferences and can’t understand why they would be excluded from the CME requirement.
I appreciate your dilemma and hard work, thanks for the opportunity to voice my opinion.