Published online Mar 21, 2020. doi: 10.3748/wjg.v26.i11.1221
Peer-review started: December 5, 2019
First decision: January 16, 2020
Revised: February 10, 2020
Accepted: March 5, 2020
Article in press: March 5, 2020
Published online: March 21, 2020
System based practice (SBP) milestones require trainees to effectively navigate the larger health care system for optimal patient care. In gastroenterology training programs, the assessment of SBP is difficult due to high volume, high acuity inpatient care, as well as inconsistent direct supervision. Nevertheless, structured assessment is required for training programs. We hypothesized that objective structured clinical examination (OSCE) would be an effective tool for assessment of SBP.
To develop a novel method for SBP milestone assessment of gastroenterology fellows using the OSCE.
For this observational study, we created 4 OSCE stations: Counseling an impaired colleague, handoff after overnight call, a feeding tube placement discussion, and giving feedback to a medical student on a progress note. Twenty-six first year fellows from 7 programs participated. All fellows encountered identical case presentations. Checklists were completed by trained standardized patients who interacted with each fellow participant. A report with individual and composite scores was generated and forwarded to program directors to utilize in formative assessment. Fellows also received immediate feedback from a faculty observer and completed a post-session program evaluation survey.
Survey response rate was 100%. The average composite score across SBP milestones for all cases were 6.22 (SBP1), 4.34 (SBP2), 3.35 (SBP3), and 6.42 (SBP4) out of 9. The lowest composite score was in SBP 3, which asks fellows to advocate for cost effective care. This highest score was in patient care 2, which asks fellows to develop comprehensive management plans. Discrepancies were identified between the fellows’ perceived performance in their self-assessments and Standardized Patient checklist evaluations for each case. Eighty-seven percent of fellows agreed that OSCEs are an important component of their clinical training, and 83% stated that the cases were similar to actual clinical encounters. All participating fellows stated that the immediate feedback was “very useful.” One hundred percent of the fellows stated they would incorporate OSCE learning into their clinical practice.
OSCEs may be used for standardized evaluation of SBP milestones. Trainees scored lower on SBP milestones than other more concrete milestones. Training programs should consider OSCEs for assessment of SBP.
Core tip: In United States medical training, system based practice (SBP) milestones are often considered the most difficult to both teach and assess. While the objective standardized clinical examination is a well validated method for assessment in medical education, its use for assessment of specific SBP milestones has not been well studied. In this observational study, we created and implemented objective standardized clinical examinations geared towards assessment of SBP milestones in gastroenterology fellows in scenarios engineered to provide opportunity for medical error. We show that this method provides objective assessment of trainees for program use and may help trainees feel more prepared for real world situations.