Observational Study
Copyright ©The Author(s) 2020. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastroenterol. Mar 21, 2020; 26(11): 1221-1230
Published online Mar 21, 2020. doi: 10.3748/wjg.v26.i11.1221
Subtle skills: Using objective structured clinical examinations to assess gastroenterology fellow performance in system based practice milestones
Marianna Papademetriou, Gabriel Perrault, Max Pitman, Colleen Gillespie, Sondra Zabar, Elizabeth Weinshel, Renee Williams
Marianna Papademetriou, Division of Gastroenterology, Georgetown University Medical Center, Washington, DC 20007, United States
Marianna Papademetriou, Division of Gastroenterology, Washington DC VA Medical Center, Washington, DC 20422, United States
Gabriel Perrault, Department of Medicine, New York University Medical Center, New York, NY 10016, United States
Max Pitman, Renee Williams, Division of Gastroenterology, New York University Medical Center, New York, NY 10016, United States
Colleen Gillespie, Sondra Zabar, Department of Medicine, New York University School of Medicine, New York, NY 10016, United States
Elizabeth Weinshel, Department of Gastroenterology, VA New York Harbor Healthcare System, New York, NY 10010, United States
Author contributions: Papademetriou M, Pitman M, Williams R and Weinshel E wrote and developed content for the Objective Structured Clinical Exam; Gillespie C, Zabar S and Weinshel E served as content experts who reviewed checklists, exam content, scripts and helped coordinate and execution of the Objective Structured Clinical Examinations; Williams R collected and summarized the data. All authors contributed in the writing and editing of the final manuscript.
Institutional review board statement: This program was considered an educational performance improvement project by the New York University School of Medicine Institutional Review Board and was not considered for IRB approval. IRB submission and approval was not required.
Informed consent statement: This program was considered an educational performance improvement project by the New York University School of Medicine Institutional Review Board. Participation in the program was voluntary. Informed consent to participate was not required.
Conflict-of-interest statement: The authors have no potential conflicts of interest to disclose.
Data sharing statement: No additional data are available.
STROBE statement: Authors have read the STROBE Statement-checklist of items and the manuscript was prepared and revised accordingly.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Corresponding author: Marianna Papademetriou, MD, Assistant Professor, Division of Gastroenterology, Washington DC VA Medical Center, 50 Irving Street NW, Washington, DC 20422, United States. marianna.papademetriou@va.gov
Received: December 5, 2019
Peer-review started: December 5, 2019
First decision: January 16, 2020
Revised: February 10, 2020
Accepted: March 5, 2020
Article in press: March 5, 2020
Published online: March 21, 2020
Processing time: 106 Days and 9.4 Hours
Abstract
BACKGROUND

System based practice (SBP) milestones require trainees to effectively navigate the larger health care system for optimal patient care. In gastroenterology training programs, the assessment of SBP is difficult due to high volume, high acuity inpatient care, as well as inconsistent direct supervision. Nevertheless, structured assessment is required for training programs. We hypothesized that objective structured clinical examination (OSCE) would be an effective tool for assessment of SBP.

AIM

To develop a novel method for SBP milestone assessment of gastroenterology fellows using the OSCE.

METHODS

For this observational study, we created 4 OSCE stations: Counseling an impaired colleague, handoff after overnight call, a feeding tube placement discussion, and giving feedback to a medical student on a progress note. Twenty-six first year fellows from 7 programs participated. All fellows encountered identical case presentations. Checklists were completed by trained standardized patients who interacted with each fellow participant. A report with individual and composite scores was generated and forwarded to program directors to utilize in formative assessment. Fellows also received immediate feedback from a faculty observer and completed a post-session program evaluation survey.

RESULTS

Survey response rate was 100%. The average composite score across SBP milestones for all cases were 6.22 (SBP1), 4.34 (SBP2), 3.35 (SBP3), and 6.42 (SBP4) out of 9. The lowest composite score was in SBP 3, which asks fellows to advocate for cost effective care. This highest score was in patient care 2, which asks fellows to develop comprehensive management plans. Discrepancies were identified between the fellows’ perceived performance in their self-assessments and Standardized Patient checklist evaluations for each case. Eighty-seven percent of fellows agreed that OSCEs are an important component of their clinical training, and 83% stated that the cases were similar to actual clinical encounters. All participating fellows stated that the immediate feedback was “very useful.” One hundred percent of the fellows stated they would incorporate OSCE learning into their clinical practice.

CONCLUSION

OSCEs may be used for standardized evaluation of SBP milestones. Trainees scored lower on SBP milestones than other more concrete milestones. Training programs should consider OSCEs for assessment of SBP.

Keywords: Objective structured clinical exams; Medical education; Medical error; System based practice; Milestones; Gastroenterology

Core tip: In United States medical training, system based practice (SBP) milestones are often considered the most difficult to both teach and assess. While the objective standardized clinical examination is a well validated method for assessment in medical education, its use for assessment of specific SBP milestones has not been well studied. In this observational study, we created and implemented objective standardized clinical examinations geared towards assessment of SBP milestones in gastroenterology fellows in scenarios engineered to provide opportunity for medical error. We show that this method provides objective assessment of trainees for program use and may help trainees feel more prepared for real world situations.