BACKGROUND AND PURPOSE:

• US internal medicine residents are assessed with milestone assessments and American Board of Internal Medicine (ABIM) internal medicine certification examination
• The relationship between resident graduates’ scores on these two assessment metrics and their patients’ outcomes is not well understood
• Gray et al. (JAMA Network Open, 2024) examined the association between physicians’ milestone ratings and certification examination scores and hospital outcomes for their patients

METHODS:

• Retrospective cohort analysis
• Population
  • Hospitalists completing training in US hospitals in 2016 to 2018
  • Caring for Medicare fee-for-service beneficiaries during hospitalizations in 2017 to 2019
• Exposures
  • Certification examination score quartile
  • Milestone ratings
    • Overall core competency rating, defined as
      • Mean of the end of residency milestone subcompetency ratings (Low | Medium | High)
    • Knowledge core competency measure (same categories)
• Study design
  • Overall core competency defined as
    • Mean of the end of residency milestone subcompetency ratings (Low | Medium | High)
  • Analyses accounted for hospital fixed effects
  • Adjustments: Patient characteristics | Physician years of experience | Year
• Primary outcomes
  • Patient 7-day mortality and readmission rates
• Secondary outcomes
  • Patient 30-day mortality and readmission rates | Length of stay | Subspecialist consultation frequency

RESULTS:

• 455,120 hospitalizations
  • Median patient age: 79 years (IQR, 73 to 86) | Female: 56.5%
• A high vs low overall or knowledge milestone core competency rating was not associated with any of the outcome measures
  • E.g. 7.-day mortality
    • 2.7% nonsignificant increase among high competency rated physicians (95% CI, −5.2 to 10.6) | P=0.51
• Top vs bottom examination score quartile was associated with differences in patients’ outcomes
  • 7-day mortality rates
    • 8.0% reduction among top scoring physicians (95% CI, –13.0 to –3.1) | P=0.002
  • 7-day readmission rates
    • 9.3% reduction among top scoring physicians (95% CI, _6.7 to –0.4) | P=0.03
• Top examination score quartile was also associated with more consultations
  • 2.4% more among top scoring physicians (95% CI, 0.8 to 3.9) | P<0.003
• There was no association between length of stay or 30-day readmission rates

CONCLUSION:

• For recent graduates of internal medicine residency programs, higher certification examination score was associated with improved patient outcomes
• Residency milestone ratings, on the other hand, were not associated with differences in patient outcomes
• The authors state

Among newly trained hospitalists, certification examination score, but not residency milestone ratings, was associated with improved outcomes among hospitalized Medicare beneficiaries, suggesting that formally incorporating standardized examinations into residency ratings might improve their validity

Learn More – Primary Sources:

Associations of Internal Medicine Residency Milestone Ratings and Certification Examination Scores With Patient Outcomes