Observational Study
Copyright ©The Author(s) 2020. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastroenterol. Mar 21, 2020; 26(11): 1221-1230
Published online Mar 21, 2020. doi: 10.3748/wjg.v26.i11.1221
Subtle skills: Using objective structured clinical examinations to assess gastroenterology fellow performance in system based practice milestones
Marianna Papademetriou, Gabriel Perrault, Max Pitman, Colleen Gillespie, Sondra Zabar, Elizabeth Weinshel, Renee Williams
Marianna Papademetriou, Division of Gastroenterology, Georgetown University Medical Center, Washington, DC 20007, United States
Marianna Papademetriou, Division of Gastroenterology, Washington DC VA Medical Center, Washington, DC 20422, United States
Gabriel Perrault, Department of Medicine, New York University Medical Center, New York, NY 10016, United States
Max Pitman, Renee Williams, Division of Gastroenterology, New York University Medical Center, New York, NY 10016, United States
Colleen Gillespie, Sondra Zabar, Department of Medicine, New York University School of Medicine, New York, NY 10016, United States
Elizabeth Weinshel, Department of Gastroenterology, VA New York Harbor Healthcare System, New York, NY 10010, United States
Author contributions: Papademetriou M, Pitman M, Williams R and Weinshel E wrote and developed content for the Objective Structured Clinical Exam; Gillespie C, Zabar S and Weinshel E served as content experts who reviewed checklists, exam content, scripts and helped coordinate and execution of the Objective Structured Clinical Examinations; Williams R collected and summarized the data. All authors contributed in the writing and editing of the final manuscript.
Institutional review board statement: This program was considered an educational performance improvement project by the New York University School of Medicine Institutional Review Board and was not considered for IRB approval. IRB submission and approval was not required.
Informed consent statement: This program was considered an educational performance improvement project by the New York University School of Medicine Institutional Review Board. Participation in the program was voluntary. Informed consent to participate was not required.
Conflict-of-interest statement: The authors have no potential conflicts of interest to disclose.
Data sharing statement: No additional data are available.
STROBE statement: Authors have read the STROBE Statement-checklist of items and the manuscript was prepared and revised accordingly.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Corresponding author: Marianna Papademetriou, MD, Assistant Professor, Division of Gastroenterology, Washington DC VA Medical Center, 50 Irving Street NW, Washington, DC 20422, United States. marianna.papademetriou@va.gov
Received: December 5, 2019
Peer-review started: December 5, 2019
First decision: January 16, 2020
Revised: February 10, 2020
Accepted: March 5, 2020
Article in press: March 5, 2020
Published online: March 21, 2020
Research background

Medical education assessment in the United States is currently based on six competencies as defined by the Accreditation Council for Graduate Medical Education. Gastroenterology (GI) training programs must assess trainees according in all six competencies as part of formative feedback. The system based practice (SBP) competencies require trainees to effectively recognize and navigate the larger healthcare system for optimal patient care. These milestones may be considered more subtle skills to master than other concrete milestones such as patient care and medical knowledge.

Research motivation

SBP milestones are difficult to observe and assess in daily clinical encounters between patients and trainees. Therefore, alterative objective activities may be necessary to adequately assess achievement in SBP. Simulation based medical education such as the Objective Structured Clinical Examinations (OSCEs) are now a standard methodology for assessing clinical skill and knowledge in medical education. OSCEs have not been previously used to assess SBP milestones specifically.

Research objectives

The main objective of this research study was to create and then evaluate OSCEs as an effective assessment tool for the evaluation of SBP milestones. We aimed to see if new clinical scenarios commonly encountered by GI trainees would be useful in this assessment. We also sought to evaluate how trainees felt about the experience.

Research methods

We developed four cases to help assess the Accreditation Council for Graduate Medical Education milestones with a focus on SBP. Trainees went through these four simulations with standardized patients and were evaluated by faculty experts using standardized checklists. Their performance from the checklists were aggregated and used to produce a scorecard which was sent to program directors at the conclusion of the OSCE. The trainees were then given direct feedback from the standardized patients and the faculty observer. Finally, the trainees were asked to complete a survey on the experience.

Research results

We ran three OSCE sessions involving 26 GI trainees. Scorecards indicated that, on average, trainees scored lower on SBP milestones than on other milestones categories. We identified and reported discrepancies between how well trainees believed they achieved objectives, and how they were rated by the standardized patients and faculty observers. Overall, trainees reflected positively on the experience in the post participation survey. They universally felt that the immediate feedback was useful and would improve their clinical skills. All participants stated they would recommend this OSCE as an assessment and training tool.

Research conclusions

In this study we demonstrated that OSCEs may be utilized to assess SBP milestones in an objective manner. Since SBP milestones may be difficult to assess in day-to-day activities in the hospital or clinic setting, training programs may want to utilize this type of standardized case-based simulation for assessment. Likewise, trainees reflected positively on the experience and felt they would incorporate feedback into their daily practice.

Research perspectives

Future studies are needed to assess if OSCEs may be useful teaching tools for SBP milestones. This would require repeat assessment with the same OSCE at the GI fellows’ completion of training and comparison of this group to a group who did not participate in the initial OSCE in their first year.