In July 2016, new enhancements to the ABIM FasTrack system for recording residents’ competency ratings, which help to determine eligibility for ABIM’s board certification exam, will go live. The ABIM FasTrack system was created at the request of the GME community, and this is the third year that milestones data has been provided to ABIM by ACGME.
Last year, functionality was added that enabled the importing of resident ratings data directly from the ACGME’s Accreditation Data System (ADS) into the ABIM FasTrack system, which alleviated the need for internal medicine program directors and administrators to enter what is essentially the same information into two different databases. Now, evaluations from prior years can be entered, for tracking residents’ ratings and learning trajectories over time.
This easing of an administrative burden for internal medicine residency programs comes at a time when the underlying framework for determining and reporting on residents’ competencies has become substantially more complex.
ABIM FasTrack has its roots in a major initiative – begun in 2009 and still ongoing – to revamp the accreditation system for medical residency programs across all ABMS specialties. The Next Accreditation System (NAS) was motivated initially by public pressures for the medical education community to do a better job of preparing physicians for the demands of contemporary medical practice. This includes (but is not limited to):
- Ensuring strong capabilities for working and leading in team-based clinical settings.
- Promoting technological literacy and capacities for exploiting technology to improve patient care and medical outcomes.
- Being more sensitive to costs in medical practice.
- Being capable of engaging patients effectively in their own care decisions.
Two key elements of the NAS — core competencies and milestones — were developed as a way to shape and evaluate the education of residents as well as measure and report on outcomes but increased the amount of paperwork and recording required of residency program directors. ABIM FasTrack is intended to help streamline the process.
Milestones and ACGME Core Competencies
The ACGME and ABMS first collaborated to define the six core competencies in 1999, which aimed to define the foundational skills every physician should possess and in turn, inform the creation of educational programs that reflect the skills and attributes that are directly relevant to patient care. What followed from the work on the core competencies was the creation and implementation of a detailed and far-reaching set of criteria – called milestones – for tracking each resident’s progress toward achieving competency in the six core areas: patient care and procedural skills, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. The milestones portion of the NAS represents an ambitious attempt to add specificity around what it means to achieve the six core competencies across very different medical specialties.
For each ABMS specialty area, a Milestone Group, with representation from key stakeholders (including residents), was established. Their work was to codify how post-graduate programs would evaluate residents’ achievements in each year of training within the six-competency framework through a detailed series of developmental levels, ranging from novice to expert/master. For internal medicine, the detailed milestones were first released in 2012; they were described at the time as:
…designed for programs to use in semi-annual review of resident performance and reporting to the ACGME. Milestones are knowledge, skills, attitudes, and other attributes for each of the ACGME competencies that describe the development of competence from an early learner up to and beyond that expected for unsupervised practice.
The Milestones at Work
The ACGME and ABIM have made it clear that milestones on their own are not used for high stakes decisions such as individual certification or program accreditation. The ACGME will use the data – primarily in the aggregate – as:
… a source of specialty-specific normative data for the specialty Review Committees to use in assessing the quality of residency and fellowship programs and facilitating improvements to program curriculum and resident and fellow performance if and when needed. The milestones will also be used by the ACGME to demonstrate accountability of the effectiveness of graduate medical education within ACGME-accredited programs in meeting the needs of the public.
ABIM, along with other internal medicine boards and organizations, participated in developing the milestones for internal medicine. The board provides information regarding the NAS milestones on its website in the context of available assessment tools, and states that data reported by residency programs into the ACGME’s Accreditation Data System (ADS) will be imported into the new ABIM FasTrack system for program directors and administrators to simply verify.
What is less clear at this time is whether (and precisely how) medical residency programs and ABMS certification boards will use the milestones data to inform residents’ eligibility for such things as program graduation and board certification. For example, will internal medicine residency programs directly correlate specific (numerical) milestone achievements to the ‘Yes’, ‘No’, and ‘Conditional on Improvement’ ratings that currently help to determine a resident’s eligibility to sit for board certification exams? If yes, will those mappings be standardized and consistent from one internal medical residency program to the next?
There does appear to be an expectation that the milestones data becomes an important part of residency programs’ processes for determining eligibility for both graduation and board certification. Under the NAS, for example, each residency program was expected to form and train (by June 2013) a Clinical Competency Committee that would utilize milestone and other data points to make consensus decisions around the progress of each resident.
As far as board certification, in a statement released in April 2013, the Education Redesign Advisory Board endorsed the release of the Internal Medicine Milestones for use in the ACGME NAS. The Advisory Board included representation from the Alliance for Academic Internal Medicine, ABIM, ACGME, the American College of Physicians, the American Medical Association, the Association of Specialty Professors, the Society of General Internal Medicine, and the Society of Hospital Medicine.
While the statement noted that a “potential use” for the milestones would be for certification boards to “ascertain whether individuals have demonstrated qualifications needed to sit for Board exams,” it is apparent that such determinations would not be made based on milestone data alone, and not outside the context of program director assessments of resident competence and overall clinical competence, as well as the assessments of competency committees, physician faculty who are directly supervising residents, fellows, et al.
That same statement notes, however, that,
While fully supportive of the milestones, the advisory board believes that defining the milestones is only the first step in a series of necessary work to realize the potential of a competency-based model of GME.
Among a list of additional steps needed, the advisory board included:
Active study of the milestones to ensure the described developmental progression of competency actually defines the desired outcomes of physicians who are capable of shaping and providing the health care needed in our complex and evolving health care system.
At least one such study focused on use of the milestones in internal medicine was published in May 2016, with data submitted by program directors at the end of the 2013-2014 academic year. This study was limited to the first reporting period in which milestones were used, and there is sure to be ongoing research into the impact and validity of the milestones. For the time being, at least, the ACGME appears disinclined to mandate how residency programs use milestones data in determining graduation eligibility. For example, the ACGME’s FAQ on the subject of program graduation criteria states that:
The ACGME has no required minimums for Milestone reporting. The determination of an individual’s readiness for graduation is at the discretion of the program director.
In the same FAQ, the ACGME emphasizes the intended use and limits of the Milestones as well as the roles of faculty members, residency programs, and the ABMS member boards:
The Milestones do not define the totality of competence or of a discipline. Judgment on the part of faculty members and the programs is and will remain essential in producing the “whole physician.” The ACGME will use the Milestones to promote better curriculum and assessment, and as one method of assessing whether programs are adequately preparing individuals for the unsupervised practice of the specialty. … In addition, the ABMS member boards will continue to assess individuals for their acquisition of the knowledge, skills, and attitudes necessary for the unsupervised practice of the specialty.
The ABIM FasTrack System: Four Things to Know
- Individual evaluation data on residents is imported directly from ACGME’s ADS, substantially lightening easing data-entry burdens on program directors and administrators.
- A former nine-point rating scale on the six competencies changes to a Yes/No/Conditional on Improvement judgment about competence.
- The former “Marginal” rating for overall clinical competence changes to “Conditional on Improvement.”
- And Moral and Ethical behavior will now be addressed under both Professional competency and in the overall competency ratings.
We would love to hear from internal medicine residency program directors, educators, and residents on what you think of the ACGME competency milestones now that they have been in active use for several years. Do they hit the mark in terms of ensuring readiness for unsupervised practice and eligibility for to assume the mantle of board certification? Do they succeed in reflecting a more outcomes-driven approach to medical education? Let us know in your comments below.
Editor’s note: this post was edited on July 1 to provide more details about about how milestones data will be used and to correct an error about the nature of the enhancements to the ABIM FasTrack system.