1
|
Steinhagen E, Stein SL, Ofshteyn A, Sherman KL, Miller-Ocuin JL, Fennern EB, Bordeianou LG, OPTIONN Expert Consensus Panel. Optimal Pouch Training: Investigating Operative and Nonoperative Needs Study. Dis Colon Rectum 2025; 68:764-775. [PMID: 40044400 DOI: 10.1097/dcr.0000000000003685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/27/2025]
Abstract
BACKGROUND Ileal pouch anal anastomosis is an important part of colorectal surgery training. However, decreasing case numbers create fewer chances to teach the knowledge and skills to trainees. OBJECTIVE The purpose of this study is to define the components and format of an adjunctive curriculum for teaching colon and rectal surgery residents ileal pouch anal anastomosis. More specifically, it aims to create expert consensus on the essential and unique elements of a curriculum, determine which alternative strategies will be acceptable, and identify methods of assessment to evaluate the effectiveness of the curriculum. DESIGN This was a modified Delphi Consensus study. Semistructured interviews with thematic analysis were used to develop the initial round of the survey. SETTING Interviews, surveys, and online consensus conference. PARTICIPANTS Faculty who were at North American Accreditation Council for Graduate Medical Education colorectal residencies, and participants from each round were invited to subsequent rounds. MAIN OUTCOME MEASURES Three rounds of surveys were used to select high-priority items for inclusion in the curriculum, educational strategies, and assessments. This was followed by a consensus conference to clarify prioritization and acceptable options for teaching and assessment. Participants from the third round of surveys, as well as Program Directors from colorectal residencies, were invited to the consensus conference. RESULTS Twelve semistructured interviews defined the educational needs of colorectal residents, the appropriateness of alternative educational strategies, and concerns about case minimums as an assessment of readiness for practice for this procedure. The needs were divided into preoperative, intraoperative, and postoperative phases, with knowledge and skills in each phase. The resulting survey was completed by 85 colorectal surgeons in round 1, 43 in round 2, and 38 in round 3. These results were subsequently summarized by a team of 9 participants in a final consensus conference, where the participants made suggestions on the recommended educational strategies for each educational need. LIMITATIONS Inclusion of colorectal surgeons involved in training programs only, bias in respondents, and participation rate. CONCLUSIONS This shared understanding of educational needs for colorectal residents that includes knowledge, judgment, and technical skills is the first step toward developing a curriculum to teach ileal pouch anal anastomosis. A collaborative effort to create the necessary curriculum is underway. See Video Abstract . ENTRENAMIENTO PTIMO EN RESERVORIO ILEAL INVESTIGACIN SOBRE LAS NECESIDADES QUIRRGICAS Y NO QUIRRGICAS ANTECEDENTES:La anastomosis ileoanal con reservorio (IPAA) es una parte importante de la formación en cirugía colorrectal. Sin embargo, la disminución en la cantidad de casos genera menos oportunidades para enseñar los conocimientos y las habilidades a los residentes.OBJETIVO:El propósito de este estudio es definir los componentes y el formato de un plan de estudios complementario para la enseñanza de la IPAA a los residentes de cirugía de colon y recto. Más específicamente, apunta a crear un consenso de expertos sobre los elementos esenciales y únicos de este plan de estudios, determinar qué estrategias alternativas serán aceptables e identificar métodos de evaluación para evaluar la efectividad del plan de estudios.DISEÑO:Este fue un estudio de consenso Delphi modificado. Se utilizaron entrevistas semiestructuradas con análisis temático para desarrollar la ronda inicial de la encuesta.ESCENARIO/PARTICIPANTES:Se invitó a los profesores de las residencias colorrectales de la ACGME de América del Norte a participar en las encuestas electrónicas, y los participantes de cada ronda fueron invitados a las rondas posteriores.MEDIDAS DE RESULTADOS:Se utilizaron tres rondas de encuestas para seleccionar elementos de alta prioridad para su inclusión en el plan de estudios, las estrategias educativas y las evaluaciones. A esto le siguió una conferencia de consenso para aclarar la priorización y las opciones aceptables para la enseñanza y la evaluación. Los participantes de la tercera ronda de encuestas, así como los directores de programas de residencias colorrectales, fueron invitados a la conferencia de consenso.RESULTADOS:Doce entrevistas semiestructuradas definieron las necesidades educativas de los residentes colorrectales, la idoneidad de las estrategias educativas alternativas y las preocupaciones sobre los mínimos de casos como una evaluación de la preparación para la práctica de este procedimiento. Las necesidades se dividieron en fases preoperatorias, intraoperatorias y posoperatorias con conocimientos y habilidades en cada una. La encuesta resultante fue completada por 85 cirujanos colorrectales en la Ronda 1, 43 en la Ronda 2 y 38 en la Ronda 3. Estos resultados fueron resumidos posteriormente por un equipo de 9 participantes en una conferencia de consenso final donde los participantes hicieron sugerencias sobre las estrategias educativas recomendadas para cada necesidad educativa.LIMITACIONES:Inclusión únicamente de cirujanos colorrectales involucrados en programas de entrenamiento; sesgo en los encuestados; tasa de participación.CONCLUSIONES:Esta comprensión compartida de las necesidades educativas de los residentes colorrectales que incluye conocimientos, criterio y habilidades técnicas es el primer paso hacia el desarrollo de un plan de estudios para enseñar IPAA. Se está llevando a cabo un esfuerzo colaborativo para crear el plan de estudios necesario. (Traducción-Dr. Jorge Silva Velazco ).
Collapse
Affiliation(s)
- Emily Steinhagen
- Department of Surgery, University Hospitals Cleveland Medical Center, Cleveland, Ohio
- University Hospitals Research in Surgical Outcomes and Effectiveness Center (UH-RISES), Cleveland, Ohio
- Department of Medical Education, Case Western School of Medicine, Cleveland, Ohio
| | - Sharon L Stein
- Department of Surgery, University Hospitals Cleveland Medical Center, Cleveland, Ohio
- University Hospitals Research in Surgical Outcomes and Effectiveness Center (UH-RISES), Cleveland, Ohio
- Department of Medical Education, Case Western School of Medicine, Cleveland, Ohio
| | - Asya Ofshteyn
- Department of Surgery, University Hospitals Cleveland Medical Center, Cleveland, Ohio
- University Hospitals Research in Surgical Outcomes and Effectiveness Center (UH-RISES), Cleveland, Ohio
- Department of Medical Education, Case Western School of Medicine, Cleveland, Ohio
| | - Karen L Sherman
- Department of Surgery, Duke University Health System, Durham, North Carolina
| | - Jennifer L Miller-Ocuin
- Department of Surgery, University Hospitals Cleveland Medical Center, Cleveland, Ohio
- University Hospitals Research in Surgical Outcomes and Effectiveness Center (UH-RISES), Cleveland, Ohio
- Department of Medical Education, Case Western School of Medicine, Cleveland, Ohio
| | - Erin B Fennern
- Department of Surgery, University Hospitals Cleveland Medical Center, Cleveland, Ohio
- University Hospitals Research in Surgical Outcomes and Effectiveness Center (UH-RISES), Cleveland, Ohio
- Department of Medical Education, Case Western School of Medicine, Cleveland, Ohio
| | - Liliana G Bordeianou
- Department of Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | | |
Collapse
|
2
|
Desir A, Pourghaderi P, Hegde SR, Demirel D, Pogacnik JS, De S, Fleshman JW, Sankaranarayanan G. Validity of task-specific metrics for assessment in perineal proctectomy. Surg Endosc 2024; 38:5319-5330. [PMID: 39026007 PMCID: PMC11365785 DOI: 10.1007/s00464-024-11029-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Accepted: 06/30/2024] [Indexed: 07/20/2024]
Abstract
BACKGROUND Perineal proctectomy is a complex procedure that requires advanced skills. Currently, there are no simulators for training in this procedure. As part of our objective of developing a virtual reality simulator, our goal was to develop and validate task-specific metrics for the assessment of performance for this procedure. We conducted a three-phase study to establish task-specific metrics, obtain expert consensus on the appropriateness of the developed metrics, and establish the discriminant validity of the developed metrics. METHODS In phase I, we utilized hierarchical task analysis to formulate the metrics. In phase II, a survey involving expert colorectal surgeons determined the significance of the developed metrics. Phase III was aimed at establishing the discriminant validity for novices (PGY1-3) and experts (PGY4-5 and faculty). They performed a perineal proctectomy on a rectal prolapse model. Video recordings were independently assessed by two raters using global ratings and task-specific metrics for the procedure. Total scores for both metrics were computed and analyzed using the Kruskal-Wallis test. A Mann-Whitney U test with Benjamini-Hochberg correction was used to evaluate between-group differences. Spearman's rank correlation coefficient was computed to assess the correlation between global and task-specific scores. RESULTS In phase II, a total of 23 colorectal surgeons were recruited and consensus was obtained on all the task-specific metrics. In phase III, participants (n = 22) included novices (n = 15) and experts (n = 7). There was a strong positive correlation between the global and task-specific scores (rs = 0.86; P < 0.001). Significant between-group differences were detected for both global (χ2 = 15.38; P < 0.001; df = 2) and task-specific (χ2 = 11.38; P = 0.003; df = 2) scores. CONCLUSIONS Using a biotissue rectal prolapse model, this study documented high IRR and significant discriminant validity evidence in support of video-based assessment using task-specific metrics.
Collapse
Affiliation(s)
- Alexis Desir
- Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Poya Pourghaderi
- Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Shruti R Hegde
- Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Doga Demirel
- Florida Polytechnic University, Lakeland, FL, USA
| | | | - Suvranu De
- Florida A&M University-Florida State University College of Engineering, Tallahassee, FL, USA
| | | | | |
Collapse
|
3
|
Nigam A, Kellam JF, Ambrose CG, Tai BL. A Data-Driven Methodology to Comprehensively Assess Bone Drilling Using Radar Plots. JB JS Open Access 2024; 9:e23.00069. [PMID: 38188189 PMCID: PMC10758530 DOI: 10.2106/jbjs.oa.23.00069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/09/2024] Open
Abstract
Background The study aims to develop a data-driven methodology to assess bone drilling in preparation for future clinical trials in residency training. The existing assessment methods are either subjective or do not consider the interdependence among individual skill factors, such as time and accuracy. This study uses quantitative data and radar plots to visualize the balance of the selected skill factors. Methods In the experiment, straight vertical drilling was assessed across 3 skill levels: expert surgeons (N = 10), intermediate residents (postgraduate year-2-5, N = 5), and novice residents (postgraduate year-1, N = 10). Motion and force were measured for each drilling trial, and data from multiple trials were then converted into 5 performance indicators, including overshoot, drilling time, overshoot consistency, time consistency, and force fluctuation. Each indicator was then scored between 0 and 10, with 10 being the best, and plotted into a radar plot. Results Statistical difference (p < 0.05) was confirmed among 3 skill levels in force, time, and overshoot data. The radar plots revealed that the novice group exhibited the most distorted pentagons compared with the well-formed pentagons observed in the case of expert participants. The intermediate group showed slight distortion that was between the expert and novice groups. Conclusion/Clinical Relevance This research shows the utility of radar plots in drilling assessment in a comprehensive manner and lays the groundwork for a data-driven training scheme to prepare novice residents for clinical practice.
Collapse
Affiliation(s)
- Aman Nigam
- Department of Mechanical Engineering, Texas A&M University, College Station, Texas
| | - James F. Kellam
- Department of Orthopaedic Surgery, University of Texas Health Science Center at Houston (UTHealth), Houston, Texas
| | - Catherine G. Ambrose
- Department of Orthopaedic Surgery, University of Texas Health Science Center at Houston (UTHealth), Houston, Texas
| | - Bruce L. Tai
- Department of Mechanical Engineering, Texas A&M University, College Station, Texas
| |
Collapse
|
4
|
Bapna T, Valles J, Leng S, Pacilli M, Nataraja RM. Eye-tracking in surgery: a systematic review. ANZ J Surg 2023; 93:2600-2608. [PMID: 37668263 DOI: 10.1111/ans.18686] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 08/20/2023] [Accepted: 08/22/2023] [Indexed: 09/06/2023]
Abstract
BACKGROUND Surgery is constantly evolving with the assistance of rapidly developing novel technology. Eye-tracking devices provide opportunities to monitor the acquisition of surgical skills, gain insight into performance, and enhance surgical practice. The aim of this review was to consolidate the available evidence for the use of eye-tracking in the surgical disciplines. METHODS A systematic literature review was conducted in accordance with PRISMA guidelines. A search of OVID Medline, EMBASE, Cochrane library, Scopus, and Science Direct was conducted January 2000 until December 2022. Studies involving eye-tracking in surgical training, assessment and technical innovation were included in the review. Non-surgical procedures, animal studies, and studies not involving surgical participants were excluded from the review. RESULTS The search returned a total of 12 054 articles, 80 of which were included in the final analysis and review. Seventeen studies involved eye-tracking in surgical training, 48 surgical assessment, and 20 were focussing on technical aspects of this technology. Twenty-six different eye-tracking devices were used in the included studies. Metrics such as the number of fixations, duration of fixations, dwell time, and cognitive workload were able to differentiate between novice and expert performance. Eight studies demonstrated the effectiveness of gaze-training for improving surgical skill. CONCLUSION The current literature shows a broad range of utility for a variety of eye-tracking devices in surgery. There remains a lack of standardization for metric parameters and gaze analysis techniques. Further research is required to validate its use to establish reliability and create uniform practices.
Collapse
Affiliation(s)
- Tanay Bapna
- Department of Paediatric Surgery & Surgical Simulation, Monash Children's Hospital, Melbourne, Victoria, Australia
| | - John Valles
- Department of Paediatric Surgery & Surgical Simulation, Monash Children's Hospital, Melbourne, Victoria, Australia
| | - Samantha Leng
- Department of Paediatric Surgery & Surgical Simulation, Monash Children's Hospital, Melbourne, Victoria, Australia
| | - Maurizio Pacilli
- Department of Paediatric Surgery & Surgical Simulation, Monash Children's Hospital, Melbourne, Victoria, Australia
- Departments of Paediatrics & Surgery, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia
| | - Ramesh Mark Nataraja
- Department of Paediatric Surgery & Surgical Simulation, Monash Children's Hospital, Melbourne, Victoria, Australia
- Departments of Paediatrics & Surgery, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia
| |
Collapse
|
5
|
Nasri BN, Mitchell JD, Jackson C, Nakamoto K, Guglielmi C, Jones DB. Distractions in the operating room: a survey of the healthcare team. Surg Endosc 2023; 37:2316-2325. [PMID: 36070145 PMCID: PMC9450817 DOI: 10.1007/s00464-022-09553-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2022] [Accepted: 08/08/2022] [Indexed: 11/19/2022]
Abstract
BACKGROUND Distractions during surgical procedures are associated with team inefficiency and medical error. Little is published about the healthcare provider's perception of distraction and its adverse impact in the operating room. We aim to explore the perception of the operating room team on multiple distractions during surgical procedures. METHODS A 26-question survey was administered to surgeons, anesthesia team members, nurses, and scrub technicians at our institution. Respondents were asked to identify and rank multiple distractions and indicate how each distraction might affect the flow of surgery. RESULTS There was 160 responders for a response rate of 19.18% (160/834), of which 71 (44.1%) male and 82 (50.9%) female, 48 (29.8%) surgeons, 59 (36.6%) anesthesiologists, Certified Registered Nurse Anesthetists (CRNA), and 53 (32.9%) OR nurses and scrub technicians. Responders were classified into a junior group (< 10 years of experience) and a senior group (≥ 10 years). Auditory distraction followed by equipment were the most distracting factors in the operating room. All potential auditory distractions in this survey were associated with higher percentage of certain level of negative impact on the flow of surgery except for music. The top 5 distractors belonged to equipment and environment categories. Phone calls/ pagers/ beepers and case relevant communications were consistently among the top 5 most common distractors. Case relevant communications, music, teaching, and consultation were the top 4 most perceived positive impact on the flow of surgery. Distractors with higher levels of "bothersome" rating appeared to associate with a higher level of perceived negative impact on the flow of surgery. Vision was the least distracting factor and appeared to cause minimal positive impact on the flow of surgery. CONCLUSIONS To our knowledge, this is the first survey studying perception of surgery, anesthesia, and OR staff on various distractions in the operating room. Fewer unnecessary distractions might improve the flow of surgery, improve OR teamwork, and potentially improve patient outcomes.
Collapse
Affiliation(s)
- Bao-Ngoc Nasri
- Department of Surgery, Division of Bariatric & Minimally Invasive Surgery, Beth Israel Deaconess Medical Center, Boston, MA, USA.
| | - John D Mitchell
- Department of Anesthesia, Beth Israel Deaconess Medical Center, Boston, MA, USA
- Department of Anesthesia, Pain Management, and Perioperative Medicine, Henry Ford Health, MI, USA
| | - Cullen Jackson
- Department of Anesthesia, Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Keitaro Nakamoto
- Department of Surgery, Division of Bariatric & Minimally Invasive Surgery, Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Charlotte Guglielmi
- Perioperative Service, Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Daniel B Jones
- Department of Surgery, Division of Bariatric & Minimally Invasive Surgery, Beth Israel Deaconess Medical Center, Boston, MA, USA
- Surgery, New Jersey Medical School, Newark, NJ, USA
| |
Collapse
|
6
|
Sankaranarayanan G, Parker LM, Khan A, Dials J, Demirel D, Halic T, Crawford A, Kruger U, De S, Fleshman JW. Objective metrics for hand-sewn bowel anastomoses can differentiate novice from expert surgeons. Surg Endosc 2023; 37:1282-1292. [PMID: 36180753 PMCID: PMC11335072 DOI: 10.1007/s00464-022-09584-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2022] [Accepted: 08/25/2022] [Indexed: 10/14/2022]
Abstract
BACKGROUND Assessing performance automatically in a virtual reality trainer or from recorded videos is advantageous but needs validated objective metrics. The purpose of this study is to obtain expert consensus and validate task-specific metrics developed for assessing performance in double-layered end-to-end anastomosis. MATERIALS AND METHODS Subjects were recruited into expert (PGY 4-5, colorectal surgery residents, and attendings) and novice (PGY 1-3) groups. Weighted average scores of experts for each metric item, completion time, and the total scores computed using global and task-specific metrics were computed for assessment. RESULTS A total of 43 expert surgeons rated our task-specific metric items with weighted averages ranging from 3.33 to 4.5 on a 5-point Likert scale. A total of 20 subjects (10 novices and 10 experts) participated in validation study. The novice group completed the task significantly more slowly than the experienced group (37.67 ± 7.09 vs 25.47 ± 7.82 min, p = 0.001). In addition, both the global rating scale (23.47 ± 4.28 vs 28.3 ± 3.85, p = 0.016) and the task-specific metrics showed a significant difference in performance between the two groups (38.77 ± 2.83 vs 42.58 ± 4.56 p = 0.027) following partial least-squares (PLS) regression. Furthermore, PLS regression showed that only two metric items (Stay suture tension and Tool handling) could reliably differentiate the performance between the groups (20.41 ± 2.42 vs 24.28 ± 4.09 vs, p = 0.037). CONCLUSIONS Our study shows that our task-specific metrics have significant discriminant validity and can be used to evaluate the technical skills for this procedure.
Collapse
Affiliation(s)
- Ganesh Sankaranarayanan
- Department of Surgery, UT Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, 75390, USA.
| | | | - Aimal Khan
- Vanderbilt University Medical Center, Nashville, TN, USA
| | - James Dials
- Florida Polytechnic University, Lakeland, FL, USA
| | - Doga Demirel
- Florida Polytechnic University, Lakeland, FL, USA
| | | | | | - Uwe Kruger
- Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Suvranu De
- Rensselaer Polytechnic Institute, Troy, NY, USA
| | | |
Collapse
|
7
|
Sankaranarayanan G, Parker LM, Jacinto K, Demirel D, Halic T, De S, Fleshman JW. Development and Validation of Task-Specific Metrics for the Assessment of Linear Stapler-Based Small Bowel Anastomosis. J Am Coll Surg 2022; 235:881-893. [PMID: 36102520 PMCID: PMC9669227 DOI: 10.1097/xcs.0000000000000389] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
INTRODUCTION Task-specific metrics facilitate the assessment of surgeon performance. This 3-phased study was designed to (1) develop task-specific metrics for stapled small bowel anastomosis, (2) obtain expert consensus on the appropriateness of the developed metrics, and (3) establish its discriminant validity. METHODS In Phase I, a hierarchical task analysis was used to develop the metrics. In Phase II, a survey of expert colorectal surgeons established the importance of the developed metrics. In Phase III, to establish discriminant validity, surgical trainees and surgeons, divided into novice and experienced groups, constructed a side-to-side anastomosis on porcine small bowel using a linear cutting stapler. The participants' performances were videotaped and rated by 2 independent observers. Partial least squares regression was used to compute the weights for the task-specific metrics to obtain weighted total score. RESULTS In Phase II, a total of 45 colorectal surgeons were surveyed: 28 with more than 15 years, 13 with 5 to 15 years, and 4 with less than 5 years of experience. The consensus was obtained on all the task-specific metrics in the more experienced groups. In Phase III, 20 subjects participated equally in both groups. The experienced group performed better than the novice group regardless of the rating scale used: global rating scale (p = 0.009) and the task-specific metrics (p = 0.012). After partial least squares regression, the weighted task-specific metric score continued to show that the experienced group performed better (p < 0.001). CONCLUSION Task-specific metric items were developed based on expert consensus and showed good discriminant validity compared with a global rating scale between experienced and novice operators. These items can be used for evaluating technical skills in a stapled small bowel anastomosis model.
Collapse
Affiliation(s)
| | - Lisa M Parker
- Department of Surgery, Baylor University Medical Center, Dallas, TX
| | - Kimberly Jacinto
- Department of Surgery, Baylor University Medical Center, Dallas, TX
| | - Doga Demirel
- Department of Computer Science, Florida Polytechnic University, Lakeland, FL
| | - Tansel Halic
- Department of Computer Science, University of Central Arkansas, Conway, AR
| | - Suvranu De
- Department of Mechanical, Aerospace and Nuclear Engineering, Rensselaer Polytechnic Institute, Troy, NY
| | - James W Fleshman
- Department of Surgery, Baylor University Medical Center, Dallas, TX
| |
Collapse
|
8
|
Hanrahan JG, Sideris M, Pasha T, Dedeilia A, Papalois A, Papalois V. Postgraduate Assessment Approaches Across Surgical Specialties: A Systematic Review of the Published Evidence. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:285-295. [PMID: 32889945 DOI: 10.1097/acm.0000000000003712] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
PURPOSE Ensuring competence for surgical trainees requires holistic assessment of the qualities and competencies necessary to practice safely and effectively. To determine the next steps toward achieving this aim, the authors conducted a systematic review to summarize and appraise the available evidence related to any assessment approach to postgraduate surgical training and to identify the dominant themes for assessment approaches across different specialties or countries. METHOD Medline and Embase were searched on January 10, 2019, without language or time restrictions. Any peer-reviewed study that described an assessment framework (in practice or novel) throughout postgraduate surgical training globally was included. An iterative review and thematic analysis were performed on full-text articles to determine assessment themes. Studies were then grouped by assessment themes. A tailored quality assessment of the studies included in the final analysis was conducted. Assessment themes and validity were compared across surgical specialties and countries. RESULTS From an initial 7,059 articles, 91 studies (evaluating 6,563 surgical trainees) were included in the final analysis. Ten defined assessment themes were extracted. Ten studies (11.0%) were deemed low risk of bias based on the quality assessment tool used and thus were determined to be high quality. Minor differences in assessment themes were observed between specialties and countries. Assessment themes neglected by individual surgical specialties and assessment themes that need validated assessment tools were identified. CONCLUSIONS This review highlights the low quality of evidence and fragmented efforts to develop and optimize surgical assessments. The minor differences observed demonstrate a common approach, globally and across specialties, related to surgical assessments. A paradigm shift in assessment approaches, which will require national and international collaboration, is required to optimize design and validation so that a comprehensive assessment of surgical competence can be implemented.
Collapse
Affiliation(s)
- John Gerrard Hanrahan
- J.G. Hanrahan is academic foundation doctor, Department of General Surgery, Lister Hospital, Stevenage, United Kingdom
| | - Michail Sideris
- M. Sideris is a specialty trainee in obstetrics and gynaecology, Women's Health Research Unit, Queen Mary University of London, London, United Kingdom
| | - Terouz Pasha
- T. Pasha is a final-year medical student, Faculty of Life Sciences and Medicine, King's College London, London, United Kingdom
| | - Aikaterini Dedeilia
- A. Dedeilia is a final-year medical student, School of Medicine, National and Kapodistrian University of Athens, Athens, Greece
| | - Apostolos Papalois
- A. Papalois is director, ELPEN Research & Experimental Centre, Athens, Greece
| | - Vassilios Papalois
- V. Papalois is professor of transplant surgery, Department of Surgery and Cancer, Renal Transplant Directorate, Imperial College Healthcare NHS Trust, London, United Kingdom
| |
Collapse
|
9
|
Affiliation(s)
- Elif Bilgic
- Department of Surgery, Division of Surgical Education, McGill University, McGill University Health Centre, 1650 Cedar Avenue, #D6.136, Montreal, Quebec H3G 1A4, Canada
| | - Sofia Valanci-Aroesty
- Department of Surgery, Division of Experimental Surgery, McGill University, McGill University Health Centre, 1650 Cedar Avenue, #D6.136, Montreal, Quebec H3G 1A4, Canada
| | - Gerald M Fried
- Department of Surgery, McGill University, McGill University Health Centre, 1650 Cedar Avenue, #D6.136, Montreal, Quebec H3G 1A4, Canada.
| |
Collapse
|
10
|
Caminiti MF, Driesman V, DeMontbrun S. The Oral and Maxillofacial Objective Structured Assessment of Technical Skills (OMOSATS) examination: a pilot study. Int J Oral Maxillofac Surg 2020; 50:277-284. [PMID: 32694035 DOI: 10.1016/j.ijom.2020.06.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2020] [Revised: 04/13/2020] [Accepted: 06/10/2020] [Indexed: 11/16/2022]
Abstract
The aim of this study was to develop a simulated technical skills examination specific to oral and maxillofacial surgery, assessing the validity of the test and its psychometric properties and obtaining feedback on the examination from both resident trainees and expert evaluators. The Oral and Maxillofacial Objective Structured Assessment of Technical Skills (OMOSATS) examination is a simulated procedure-based eight-station technical skills examination. Fourteen oral and maxillofacial surgery (OMFS) residents of various levels participated in 12-minute surgical tasks and were evaluated by blinded expert OMFS surgeons. The tasks included suturing, tracheostomy, arthroscopy, dental implant placement, plate bending and lag screw application, microvascular anastomosis, and skin grafting. These tasks reflect a broad range of technical milestones necessary for the practice of OMFS. Tasks were evaluated using a validated global rating scale and task-specific checklists. The results of the pilot study indicated strong face and construct validity for seven of the eight OMOSATS stations in discerning the level of training based on technical skill demonstrated at each of the eight surgical stations. Additionally, through one-way ANOVA, P-values were observed to confirm the level of discernment. The OMOSATS examination is a novel way to assess the technical skills of OMFS residents. This type of performance-based assessment has been used in other surgical domains for many years. There is strong evidence to support the use of technical skills examinations to help evaluate, teach, and provide surgeon educators with training outcomes for our OMFS trainees.
Collapse
Affiliation(s)
- M F Caminiti
- Department of Oral and Maxillofacial Surgery, Faculty of Dentistry, University of Toronto, Toronto, Ontario, Canada.
| | - V Driesman
- Department of Oral and Maxillofacial Surgery, Faculty of Dentistry, University of Toronto, Toronto, Ontario, Canada
| | - S DeMontbrun
- Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
11
|
Vaidya A, Aydin A, Ridgley J, Raison N, Dasgupta P, Ahmed K. Current Status of Technical Skills Assessment Tools in Surgery: A Systematic Review. J Surg Res 2020; 246:342-378. [DOI: 10.1016/j.jss.2019.09.006] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2019] [Revised: 08/29/2019] [Accepted: 09/11/2019] [Indexed: 12/18/2022]
|
12
|
Shahbazi M, Poursartip B, Siroen K, Schlachta CM, Patel RV. Robotics-Assisted Surgical Skills Evaluation based on Electrocortical Activity. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019; 2018:3673-3676. [PMID: 30441169 DOI: 10.1109/embc.2018.8513077] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Skills assessment in Robotics-Assisted Minimally Invasive Surgery (RAMIS) is mainly performed based on temporal, motion-based and outcome-based metrics. While these components are essential for the proper assessment of skills in RAMIS, they do not suffice for full representation of all underlying aspects of skilled performance. Besides such commonplace components of skills, there exist other elements to be taken into account for comprehensive skills assessment. Among such elements are cognitive states (such as levels of stress, attention, concentration) that can directly affect performance. Investigating the impact of electrocortical activity and cognitive states of RAMIS surgeons over their performance has, however, received little attention in the literature. Therefore, in this paper, novel performance metrics based on electroencephalography (EEG) signals are studied for potential augmentation into RAMIS training and its assessment platform. For this purpose, a user study was conducted involving 23 novices and 9 expert RAMIS surgeons. The participants were asked to perform two tasks on the dv-Trainer®, (Mimic Technologies) RAMIS simulator, while their brain EEG signals were being measured using the Muse EEG headband (InteraXon Inc.). The performance metrics were defined as mean values of band powers of EEG signals over various ranges of frequency. Statistical analysis was performed to evaluate metrics over 5 different ranges of frequency for 4 electrode locations and during 2 RAMIS training tasks. The results indicated statistically significant differences in electrocortical activity between novices and experts in temporoparietal and left frontal regions of their brain for mid to high-frequency ranges. Overall, RAMIS experts showed lower levels of electrocortical activity in those regions compared to novices. The results indicate that electrocortical activity measured by EEG signals have the potential to provide useful information for skills assessment in RAMIS.
Collapse
|
13
|
Surgical education research: How to move beyond the survey. Surgery 2019; 167:269-272. [PMID: 31253415 DOI: 10.1016/j.surg.2019.05.027] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2019] [Accepted: 05/15/2019] [Indexed: 12/22/2022]
Abstract
BACKGROUND Surgical education serves as the foundation for high-quality, efficient patient care. Because clinical knowledge is gained and newer surgical techniques developed, this material is incorporated into the curricula of surgical trainees. Methods for studying this integration vary widely, however, providing data that, at times, is challenging to interpret in the context of patient care and outcomes. METHODS A review of the literature was conducted to evaluate current practices in surgical education research, as well as topics areas of focus. RESULTS Several techniques in surgical education research currently exist, including surveys, knowledge assessments, tests of skill, and single-arm educational interventions, which may or may not include a prepost design with assessments administered both before and immediately following the intervention. The applicability of these measures to patient outcomes is variable. CONCLUSION Research in surgical education represents a field of great interest, with opportunity for novel investigations among a broad collection of topic areas. Educational research should be approached in a methodologically rigorous fashion with high investigational standards in order to advance the education of surgical trainees and the care of surgical patients.
Collapse
|
14
|
Bilgic E, Al Mahroos M, Landry T, Fried GM, Vassiliou MC, Feldman LS. Assessment of surgical performance of laparoscopic benign hiatal surgery: a systematic review. Surg Endosc 2019; 33:3798-3805. [PMID: 30671670 DOI: 10.1007/s00464-019-06662-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2018] [Accepted: 01/14/2019] [Indexed: 01/19/2023]
Abstract
BACKGROUND Operative skills correlate with patient outcomes, yet at the completion of training or after learning a new procedure, these skills are rarely formally evaluated. There is interest in the use of summative video assessment of laparoscopic benign foregut and hiatal surgery (LFS). If this is to be used to determine competency, it must meet the robust criteria established for high-stakes assessments. The purpose of this review is to identify tools that have been used to assess performance of LFS and evaluate the available validity evidence for each instrument. METHODS A systematic search was conducted up to July 2017. Eligible studies reported data on tools used to assess performance in the operating room during LFS. Two independent reviewers considered 1084 citations for eligibility. The characteristics and testing conditions of each assessment tool were recorded. Validity evidence was evaluated using five sources of validity (content, response process, internal structure, relationship to other variables, and consequences). RESULTS There were six separate tools identified. Two tools were generic to laparoscopy, and four were specific to LFS [two specific to Nissen fundoplication (NF), one heller myotomy (HM), and one paraesophageal hernia repair (PEH)]. Overall, only one assessment was supported by moderate evidence while the others had limited or unknown evidence. Validity evidence was based mainly on internal structure (all tools reporting reliability and item analysis) and content (two studies referencing previous papers for tool development in the context of clinical assessment, and four listing items without specifying the development procedures). There was little or no evidence supporting test response process (one study reporting rater training), relationship to other variables (two comparing scores in subjects with different clinical experience), and consequences (no studies). Two tools were identified to have evidence for video assessment, specific to NF. CONCLUSION There is limited evidence supporting the validity of assessment tools for laparoscopic foregut surgery. This precludes their use for summative video-based assessment to verify competency. Further research is needed to develop an assessment tool designed for this purpose.
Collapse
Affiliation(s)
- Elif Bilgic
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, Montreal, QC, Canada
| | - Mohammed Al Mahroos
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, Montreal, QC, Canada
| | - Tara Landry
- Montreal General Hospital Medical Library, McGill University Health Centre, 1650, Cedar Avenue, L9. 309, Montréal, QC, H3G 1A4, Canada
| | - Gerald M Fried
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, Montreal, QC, Canada
| | - Melina C Vassiliou
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, Montreal, QC, Canada
| | - Liane S Feldman
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, Montreal, QC, Canada.
| |
Collapse
|
15
|
Use of Simulation in High-Stakes Summative Assessments in Surgery. COMPREHENSIVE HEALTHCARE SIMULATION: SURGERY AND SURGICAL SUBSPECIALTIES 2019. [DOI: 10.1007/978-3-319-98276-2_11] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
|
16
|
Performance Assessment. COMPREHENSIVE HEALTHCARE SIMULATION: SURGERY AND SURGICAL SUBSPECIALTIES 2019. [DOI: 10.1007/978-3-319-98276-2_9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
|
17
|
Abdelsattar JM, AlJamal YN, Ruparel RK, Rowse PG, Heller SF, Farley DR. Correlation of Objective Assessment Data With General Surgery Resident In-Training Evaluation Reports and Operative Volumes. JOURNAL OF SURGICAL EDUCATION 2018; 75:1430-1436. [PMID: 29773409 DOI: 10.1016/j.jsurg.2018.04.016] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/12/2017] [Revised: 03/30/2018] [Accepted: 04/22/2018] [Indexed: 06/08/2023]
Abstract
OBJECTIVE Faculty evaluations, ABSITE scores, and operative case volumes often tell little about true resident performance. We developed an objective structured clinical examination called the Surgical X-Games (5 rooms, 15 minutes each, 12-15 tests total, different for each postgraduate [PGY] level). We hypothesized that performance in X-Games will prove more useful in identifying areas of strength or weakness among general surgery (GS) residents than faculty evaluations, ABSITE scores, or operative cases volumes. DESIGN PGY 2 to 5 GS residents (n = 35) were tested in a semiannual X-Games assessment using multiple simulation tasks: laparoscopic skills, bowel anastomosis, CT/CXR analysis, chest tube placement, etc. over 1 academic year. Resident scores were compared to their ABSITE, in-training evaluation reports, and operating room case numbers. SETTING Academic medical center. PARTICIPANTS PGY-2, 3, 4, and 5 GS residents at Mayo Clinic in Rochester, MN. RESULTS Results varied greatly within each class except for staff evaluations: in-training evaluation reports medians for PGY-2s were 5.3 (range: 5.0-6.0), PGY-3s 5.9 (5.5-6.3), PGY-4s 5.6 (5.0-6.0), and PGY-5s were 6.1 (5.6-6.9). Although ABSITE and operating room case volumes fluctated greatly with each PGY class, only X-Games scores (median: PGY-2 = 82, PGY-3 = 61, PGY-4 = 76, and PGY-5 = 60) correlated positively (p < 0.05) with operative case volume and negatively (p < 0.05) with staff evaluations. CONCLUSIONS X-Games assessment generated wide differentiation of resident performance quickly, inexpensively, and objectively. Although "Minnesota-nice" surgical staff may feel all GS trainees are "above average," objective assessment tells us otherwise.
Collapse
Affiliation(s)
- Jad M Abdelsattar
- Department of Surgery, Mayo Clinic College of Medicine, Rochester, Minnesota
| | - Yazan N AlJamal
- Department of Surgery, Mayo Clinic College of Medicine, Rochester, Minnesota
| | - Raaj K Ruparel
- Department of Surgery, Mayo Clinic College of Medicine, Rochester, Minnesota
| | - Phillip G Rowse
- Department of Surgery, Mayo Clinic College of Medicine, Rochester, Minnesota
| | - Stephanie F Heller
- Department of Surgery, Mayo Clinic College of Medicine, Rochester, Minnesota
| | - David R Farley
- Department of Surgery, Mayo Clinic College of Medicine, Rochester, Minnesota.
| |
Collapse
|
18
|
Putnam MD, Adams JE, Lender P, Van Heest AE, Shanedling JR, Nuckley DJ, Bechtold JE. Examination of Skill Acquisition and Grader Bias in a Distal Radius Fracture Fixation Model. JOURNAL OF SURGICAL EDUCATION 2018; 75:1299-1308. [PMID: 29502990 DOI: 10.1016/j.jsurg.2018.01.019] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/30/2017] [Revised: 11/07/2017] [Accepted: 01/24/2018] [Indexed: 06/08/2023]
Abstract
OBJECTIVES Primary: Assess the ability of faculty graders to predict the objectively measured strength of distal radius fracture fixation. Secondary: Compare resident skill variation and retention related to other knowable training data. DESIGN Residents were allowed 60 minutes to stabilize a standardized distal radius fracture using an assigned fixed-angle volar plate. Faculty observed and subjectively graded the residents without providing real-time feedback. Objective biomechanical evaluation (construct strength and stiffness) was compared to subjective grades. Resident-specific characteristics (sex, PGY, and ACGME case log) were also used to compare the objective data. SETTING A simulated operating room in our laboratory. PARTICIPANTS Post-graduate year 2, 3, 4, and 5 orthopedic residents. RESULTS Primary: Faculty were not successful at predicting objectively measured fixation, and their subjective scoring suggests confirmation bias as PGY increased. Secondary: Resident year-in-training alone did not predict objective measures (p = 0.53), but was predictive of subjective scores (p < 0.001). Skills learned were not always retained, as 29% of residents objectively failed subsequent to passing. Notably, resident-reported case-specific experience alone was inversely correlated with objective fixation strength. CONCLUSIONS This testing model enabled the collection of objective and subjective resident skill scores. Faculty graders did not routinely predict objective measures, and their subjective assessment appears biased related to PGY. Also, in vivo case volume alone does not predict objective results. Familiar faculty teaching consistency, and resident grading by external faculty unfamiliar with tested residents, might alter these results.
Collapse
Affiliation(s)
- Matthew D Putnam
- Department of Orthopaedic Surgery, University of Minnesota, Minneapolis, Minnesota.
| | | | - Paul Lender
- Department of Orthopaedic Surgery, University of Minnesota, Minneapolis, Minnesota
| | - Ann E Van Heest
- Department of Orthopaedic Surgery, University of Minnesota, Minneapolis, Minnesota
| | - Janet R Shanedling
- Clinical and Translational Science, University of Minnesota, Minneapolis, Minnesota
| | | | - Joan E Bechtold
- Department of Orthopaedic Surgery, University of Minnesota, Minneapolis, Minnesota; Minneapolis Medical Research Foundation and Excelen Center for Bone and Joint Education and Research, Minneapolis, Minnesota.
| |
Collapse
|
19
|
Kazan R, Viezel-Mathieu A, Cyr S, Hemmerling TM, Gilardino MS. The Montreal Augmentation Mammaplasty Operation (MAMO) Simulator: An Alternative Method to Train and Assess Competence in Breast Augmentation Procedures. Aesthet Surg J 2018; 38:835-849. [PMID: 29506205 DOI: 10.1093/asj/sjx267] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Surgical residents' exposure to aesthetic procedures remains limited in residency training. The development of the Montreal augmentation mammaplasty operation (MAMO) simulator aims to provide an adjunctive training method and assessment tool to complement the evolving competency-based surgical curriculum. OBJECTIVES To perform face, content, and construct validations of the MAMO simulator for subpectoral breast augmentation procedures and assess the reliability of the assessment scales used. METHODS Plastic surgery staff and residents were recruited to perform a subpectoral breast augmentation on the simulator. Video recordings of their performance were blindly evaluated using the objective structured assessment of technical skills (OSATS) system consisting of the global rating scale (GRS), mammaplasty objective assessment tool (MOAT), and a surgery-specific Checklist score. RESULTS Fourteen plastic surgery residents and seven expert plastic surgeons were enrolled. Experts' performance was significantly higher than residents' according to each of GRS, MOAT, and Checklist scores. Mean values of residents and experts were 23.4 (2.5) vs 36.9 (3.1) (P < 0.0001) for GRS score, 30.4 (2.2) vs 40 (3.2) (P < 0.0001) for MOAT scores, and 9.7 (1.5) vs 12 (1) (P < 0.001) for Checklist scores, respectively. Face and content validations showed excellent results among parameters evaluated, with an overall mean score of 4.8 (0.3) on 5. Cronbach's alpha was 0.96 and 0.83 for GRS and MOAT scores, respectively. Intraclass correlation coefficients for interrater reliability were excellent at 0.93, 0.92, and 0.89 for the GRS, MOAT, and Checklist scores, respectively. CONCLUSIONS This study proves the construct simulator to be valid and the assessment scales to be reliable.
Collapse
Affiliation(s)
- Roy Kazan
- Division of Plastic and Reconstructive Surgery, Department of Surgery, McGill University, Montreal, Quebec, Canada
| | - Alex Viezel-Mathieu
- Division of Plastic and Reconstructive Surgery, Department of Surgery, McGill University, Montreal, Quebec, Canada
| | - Shantale Cyr
- Intelligence Technologies of Anesthesia Group Laboratory, Department of Anesthesia, McGill University
| | | | - Mirko S Gilardino
- Division of Plastic and Reconstructive Surgery, Department of Surgery, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
20
|
Abstract
OBJECTIVE Endoscopic ear surgery (EES) is an emerging technique requiring single-handed dissection with limited depth perception. Current options for EES simulation and training are limited. Herein, we introduce a versatile, low-cost surgical skills trainer that aims to improve the fine motor control necessary for EES. STUDY DESIGN Prospective validation study. SETTING Surgical simulation laboratory. PARTICIPANTS Seven subjects ranging in experience from medical students ("Novices") to experienced ear surgeons ("Experts") participated in the validation study. Experts (n = 3) were defined as performing >10 EES cases per year. METHODS The skills trainer was constructed from a 3" diameter polyvinyl chloride pipe cap modified with two ports for instrument passage. A wooden platform was placed inside at an appropriate working distance for ear surgery. Eight interchangeable skills modules were fabricated on wooden squares (3 cm × 3 cm) using materials such as #19 wire brads, 1.6 mm glass beads, and 26-gauge jewelry wire. The material cost of this reusable model was $15. Subjects completed each skills module in triplicate, followed by a Likert-based survey. RESULTS Expert performance was superior to novices in 100% (8/8) of skills modules, i.e., threading beads on a wire (43 versus 127 s, p < 0.001) and placing a simulated prosthesis (13 versus 68 s, p = 0.01). Most participants (86%) agreed the trainer orientation was accurate and all participants (100%) were satisfied with the experience. CONCLUSIONS This low-cost modular task trainer may help fill a void in otologic training by allowing efficient, deliberate practice of validated exercises designed to improve fine motor control with EES instrumentation.
Collapse
|
21
|
de Montbrun S, Louridas M, Szasz P, Harris KA, Grantcharov TP. Developing the Blueprint for a General Surgery Technical Skills Certification Examination: A Validation Study. JOURNAL OF SURGICAL EDUCATION 2018; 75:344-350. [PMID: 28864267 DOI: 10.1016/j.jsurg.2017.08.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/10/2017] [Revised: 06/22/2017] [Accepted: 08/05/2017] [Indexed: 06/07/2023]
Abstract
INTRODUCTION There is a recognized need to develop high-stakes technical skills assessments for decisions of certification and resident promotion. High-stakes examinations requires a rigorous approach in accruing validity evidence throughout the developmental process. One of the first steps in development is the creation of a blueprint which outlines the potential content of examination. The purpose of this validation study was to develop an examination blueprint for a Canadian General Surgery assessment of technical skill certifying examination. METHODS A Delphi methodology was used to gain consensus amongst Canadian General Surgery program directors as to the content (tasks or procedures) that could be included in a certifying Canadian General Surgery examination. Consensus was defined a priori as a Cronbach's α ≥ 0.70. All procedures or tasks reaching a positive consensus (defined as ≥80% of program directors rated items as ≥4 on the 5-point Likert scale) were then included in the final examination blueprint. RESULTS Two Delphi rounds were needed to reach consensus. Of the 17 General Surgery Program directors across the country, 14 (82.4%) and 10 (58.8%) program directors responded to the first and second round, respectively. A total of 59 items and procedures reached positive consensus and were included in the final examination blueprint. CONCLUSIONS The present study has outlined the development of an examination blueprint for a General Surgery certifying examination using a consensus-based methodology. This validation study will serve as the foundational work from which simulated model will be developed, pilot tested and evaluated.
Collapse
Affiliation(s)
- Sandra de Montbrun
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada; Division of General Surgery, St. Michael's Hospital, Toronto, Ontario, Canada.
| | - Marisa Louridas
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Peter Szasz
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Kenneth A Harris
- Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada
| | - Teodor P Grantcharov
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada; Division of General Surgery, St. Michael's Hospital, Toronto, Ontario, Canada
| |
Collapse
|
22
|
Lucas BP, Tierney DM, Jensen TP, Dancel R, Cho J, El-Barbary M, Franco-Sadud R, Soni NJ. Credentialing of Hospitalists in Ultrasound-Guided Bedside Procedures: A Position Statement of the Society of Hospital Medicine. J Hosp Med 2018; 13:117-125. [PMID: 29340341 DOI: 10.12788/jhm.2917] [Citation(s) in RCA: 30] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Abstract
Ultrasound guidance is used increasingly to perform the following 6 bedside procedures that are core competencies of hospitalists: abdominal paracentesis, arterial catheter placement, arthrocentesis, central venous catheter placement, lumbar puncture, and thoracentesis. Yet most hospitalists have not been certified to perform these procedures, whether using ultrasound guidance or not, by specialty boards or other institutions extramural to their own hospitals. Instead, hospital privileging committees often ask hospitalist group leaders to make ad hoc intramural certification assessments as part of credentialing. Given variation in training and experience, such assessments are not straightforward "sign offs." We thus convened a panel of experts to conduct a systematic review to provide recommendations for credentialing hospitalist physicians in ultrasound guidance of these 6 bedside procedures. Pathways for initial and ongoing credentialing are proposed. A guiding principle of both is that certification assessments for basic competence are best made through direct observation of performance on actual patients.
Collapse
Affiliation(s)
- Brian P Lucas
- Medicine Service, White River Junction VA Medical Center, White River Junction, Vermont, USA
- Geisel School of Medicine at Dartmouth College, Hanover, New Hampshire, USA.
| | - David M Tierney
- Abbott Northwestern Hospital, Department of Medical Education, Minneapolis, Minnesota, USA
| | - Trevor P Jensen
- Division of Hospital Medicine, Department of Medicine, University of California San Francisco, San Francisco, California, USA
| | - Ria Dancel
- Division of Hospital Medicine, Department of Medicine, University of North Carolina, Chapel Hill, North Carolina, USA
| | - Joel Cho
- Division of Hospital Medicine, Kaiser Permanente San Francisco Medical Center, San Francisco, California, USA
| | - Mahmoud El-Barbary
- King Saud Bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
| | - Ricardo Franco-Sadud
- Division of General Internal Medicine, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Nilam J Soni
- Division of General & Hospital Medicine, The University of Texas School of Medicine at San Antonio, San Antonio, Texas, USA
- Section of Hospital Medicine, South Texas Veterans Health Care System, San Antonio, Texas, USA
| | | |
Collapse
|
23
|
Chen CY, Ragle CA, Lencioni R, Fransson BA. Comparison of 2 training programs for basic laparoscopic skills and simulated surgery performance in veterinary students. Vet Surg 2017; 46:1187-1197. [DOI: 10.1111/vsu.12729] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2016] [Revised: 04/21/2017] [Accepted: 06/01/2017] [Indexed: 11/30/2022]
Affiliation(s)
- Chi-Ya Chen
- Department of Veterinary Clinical Sciences; Washington State University; Pullman Washington
| | - Claude A. Ragle
- Department of Veterinary Clinical Sciences; Washington State University; Pullman Washington
| | - Rachael Lencioni
- Department of Veterinary Clinical Sciences; Washington State University; Pullman Washington
| | - Boel A. Fransson
- Department of Veterinary Clinical Sciences; Washington State University; Pullman Washington
| |
Collapse
|
24
|
Hall AK, Damon Dagnone J, Moore S, Woolfrey KGH, Ross JA, McNeil G, Hagel C, Davison C, Sebok‐Syer SS. Comparison of Simulation-based Resuscitation Performance Assessments With In-training Evaluation Reports in Emergency Medicine Residents: A Canadian Multicenter Study. AEM EDUCATION AND TRAINING 2017; 1:293-300. [PMID: 30051047 PMCID: PMC6001706 DOI: 10.1002/aet2.10055] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/23/2017] [Revised: 07/19/2017] [Accepted: 08/08/2017] [Indexed: 05/23/2023]
Abstract
OBJECTIVE Simulation stands to serve an important role in modern competency-based programs of assessment in postgraduate medical education. Our objective was to compare the performance of individual emergency medicine (EM) residents in a simulation-based resuscitation objective structured clinical examination (OSCE) using the Queen's Simulation Assessment Tool (QSAT), with portfolio assessment of clinical encounters using a modified in-training evaluation report (ITER) to understand in greater detail the inferences that may be drawn from a simulation-based OSCE assessment. METHODS A prospective observational study was employed to explore the use of a multicenter simulation-based OSCE for evaluation of resuscitation competence. EM residents from five Canadian academic sites participated in the OSCE. Video-recorded performances were scored by blinded raters using the scenario-specific QSATs with domain-specific anchored scores (primary assessment, diagnostic actions, therapeutic actions, communication) and a global assessment score (GAS). Residents' portfolios were evaluated using a modified ITER subdivided by CanMEDS roles (medical expert, communicator, collaborator, leader, health advocate, scholar, and professional) and a GAS. Correlational and regression analyses were performed comparing components of each of the assessment methods. RESULTS Portfolio review and ITER scoring was performed for 79 residents participating in the simulation-based OSCE. There was a significant positive correlation between total OSCE and ITER scores (r = 0.341). The strongest correlations were found between ITER medical expert score and each of the OSCE GAS (r = 0.420), communication (r = 0.443), and therapeutic action (r = 0.484) domains. ITER medical expert was a significant predictor of OSCE total (p = 0.002). OSCE therapeutic action was a significant predictor of ITER total (p = 0.02). CONCLUSIONS Simulation-based resuscitation OSCEs and portfolio assessment captured by ITERs appear to measure differing aspects of competence, with weak to moderate correlation between those measures of conceptually similar constructs. In a program of competency-based assessment of EM residents, a simulation-based OSCE using the QSAT shows promise as a tool for assessing medical expert and communicator roles.
Collapse
Affiliation(s)
- Andrew Koch Hall
- Department of Emergency MedicineQueen's UniversityKingstonOntarioCanada
| | - J. Damon Dagnone
- Department of Emergency MedicineQueen's UniversityKingstonOntarioCanada
| | - Sean Moore
- Department of Emergency MedicineNorthern Ontario School of MedicineKenoraOntarioCanada
| | | | - John A. Ross
- Department of Emergency MedicineDalhousie UniversityHalifaxNova ScotiaCanada
| | - Gordon McNeil
- Department of Emergency MedicineUniversity of CalgaryCalgaryAlbertaCanada
| | - Carly Hagel
- Department of Emergency MedicineQueen's UniversityKingstonOntarioCanada
| | - Colleen Davison
- Department of Emergency MedicineQueen's UniversityKingstonOntarioCanada
- Department of Public Health SciencesQueen's UniversityKingstonOntarioCanada
| | - Stefanie S. Sebok‐Syer
- Centre for Education Research & InnovationSchulich School of Medicine and DentistryWestern UniversityLondonOntarioCanada
| |
Collapse
|
25
|
Nemani A, Ahn W, Cooper C, Schwaitzberg S, De S. Convergent validation and transfer of learning studies of a virtual reality-based pattern cutting simulator. Surg Endosc 2017; 32:1265-1272. [PMID: 28812196 DOI: 10.1007/s00464-017-5802-8] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2017] [Accepted: 07/28/2017] [Indexed: 12/20/2022]
Abstract
INTRODUCTION Research has clearly shown the benefits of surgical simulators to train laparoscopic motor skills required for positive patient outcomes. We have developed the Virtual Basic Laparoscopic Skill Trainer (VBLaST) that simulates tasks from the Fundamentals of Laparoscopic Surgery (FLS) curriculum. This study aims to show convergent validity of the VBLaST pattern cutting module via the CUSUM method to quantify learning curves along with motor skill transfer from simulation environments to ex vivo tissue samples. METHODS 18 medical students at the University at Buffalo, with no prior laparoscopic surgical skills, were placed into the control, FLS training, or VBLaST training groups. Each training group performed pattern cutting trials for 12 consecutive days on their respective simulation trainers. Following a 2-week break period, the trained students performed three pattern cutting trials on each simulation platform to measure skill retention. All subjects then performed one pattern cutting task on ex vivo cadaveric peritoneal tissue. FLS and VBLaST pattern cutting scores, CUSUM scores, and transfer task completion times were reported. RESULTS Results indicate that the FLS and VBLaST trained groups have significantly higher task performance scores than the control group in both the VBLaST and FLS environments (p < 0.05). Learning curve results indicate that three out of seven FLS training subjects and four out of six VBLaST training subjects achieved the "senior" performance level. Furthermore, both the FLS and VBLaST trained groups had significantly lower transfer task completion times on ex vivo peritoneal tissue models (p < 0.05). CONCLUSION We characterized task performance scores for trained VBLaST and FLS subjects via CUSUM analysis of the learning curves and showed evidence that both groups have significant improvements in surgical motor skill. Furthermore, we showed that learned surgical skills in the FLS and VBLaST environments transfer not only to the different simulation environments, but also to ex vivo tissue models.
Collapse
Affiliation(s)
- Arun Nemani
- Rensselaer Polytechnic Institute, 110, 8th Street, Troy, NY, 12180, USA
| | - Woojin Ahn
- Rensselaer Polytechnic Institute, 110, 8th Street, Troy, NY, 12180, USA
| | - Clairice Cooper
- University at Buffalo School of Medicine and Biomedical Sciences, Buffalo, NY, USA
| | - Steven Schwaitzberg
- University at Buffalo School of Medicine and Biomedical Sciences, Buffalo, NY, USA
| | - Suvranu De
- Rensselaer Polytechnic Institute, 110, 8th Street, Troy, NY, 12180, USA.
| |
Collapse
|
26
|
de Montbrun S. Passing a Technical Skills Examination in the First Year of Surgical Residency Can Predict Future Performance. J Grad Med Educ 2017. [PMID: 28638511 PMCID: PMC5476382 DOI: 10.4300/jgme-d-16-00517.1] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND The ability of an assessment to predict performance would be of major benefit to residency programs, allowing for early identification of residents at risk. OBJECTIVE We sought to establish whether passing the Objective Structured Assessment of Technical Skills (OSATS) examination in postgraduate year 1 (PGY-1) predicts future performance. METHODS Between 2002 and 2012, 133 PGY-1 surgery residents at the University of Toronto (Toronto, Ontario, Canada) completed an 8-station, simulated OSATS examination as a component of training. With recently set passing scores, residents were assigned a pass/fail status using 3 standards setting methods (contrasting groups, borderline group, and borderline regression). Future in-training performance was compared between residents who had passed and those who failed the OSATS, using in-training evaluation reports from resident files. A Mann-Whitney U test compared performance among groups at PGY-2 and PGY-4 levels. RESULTS Residents who passed the OSATS examination outperformed those who failed, when compared during PGY-2 across all 3 standard setting methodologies (P < .05). During PGY-4, only the contrasting groups method showed a significant difference (P < .05). CONCLUSIONS We found that PGY-1 surgical resident pass/fail status on a technical skills examination was associated with future performance on in-training evaluation reports in later years. This provides validity evidence for the current PGY-1 pass/fail score, and suggests that this technical skills examination may be used to predict performance and to identify residents who require remediation.
Collapse
|
27
|
Kilkenny JJ, Singh A, Kerr CL, Khosa DK, Fransson BA. Factors associated with simulator-assessed laparoscopic surgical skills of veterinary students. J Am Vet Med Assoc 2017; 250:1308-1315. [DOI: 10.2460/javma.250.11.1308] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
28
|
Schwab B, Hungness E, Barsness KA, McGaghie WC. The Role of Simulation in Surgical Education. J Laparoendosc Adv Surg Tech A 2017; 27:450-454. [DOI: 10.1089/lap.2016.0644] [Citation(s) in RCA: 33] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Affiliation(s)
- Ben Schwab
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois
| | - Eric Hungness
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois
| | - Katherine Ann Barsness
- Division of Pediatric Surgery, Ann and Robert H Lurie Children's Hospital of Chicago, Chicago, Illinois
| | - William Craig McGaghie
- Department of Medical Education, Northwestern University Feinberg School of Medicine, Chicago, Illinois
| |
Collapse
|
29
|
Vergis A, Hardy K. Cognitive and Technical Skill Assessment in Surgical Education: a Changing Horizon. Indian J Surg 2017; 79:153-157. [PMID: 28442843 DOI: 10.1007/s12262-017-1603-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2010] [Accepted: 10/19/2010] [Indexed: 01/22/2023] Open
Abstract
Assessment is an integral component of training and credentialing surgeons for practice. Traditional methods of cognitive and technical appraisal are well established but have clear shortcomings. This review outlines the components of the surgical care assessment model, identifies the deficits of current evaluation techniques, and discusses novel and emerging technologies that attempt to ameliorate this educational void.
Collapse
Affiliation(s)
- Ashley Vergis
- Section of General Surgery, University of Manitoba, Winnipeg, MB Canada.,St. Boniface General Hospital, Z3039-409 Tache Avenue, Winnipeg, MB R2H 2A6 Canada
| | - Krista Hardy
- Section of General Surgery, University of Manitoba, Winnipeg, MB Canada
| |
Collapse
|
30
|
Vedula SS, Ishii M, Hager GD. Objective Assessment of Surgical Technical Skill and Competency in the Operating Room. Annu Rev Biomed Eng 2017; 19:301-325. [PMID: 28375649 DOI: 10.1146/annurev-bioeng-071516-044435] [Citation(s) in RCA: 79] [Impact Index Per Article: 9.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/16/2023]
Abstract
Training skillful and competent surgeons is critical to ensure high quality of care and to minimize disparities in access to effective care. Traditional models to train surgeons are being challenged by rapid advances in technology, an intensified patient-safety culture, and a need for value-driven health systems. Simultaneously, technological developments are enabling capture and analysis of large amounts of complex surgical data. These developments are motivating a "surgical data science" approach to objective computer-aided technical skill evaluation (OCASE-T) for scalable, accurate assessment; individualized feedback; and automated coaching. We define the problem space for OCASE-T and summarize 45 publications representing recent research in this domain. We find that most studies on OCASE-T are simulation based; very few are in the operating room. The algorithms and validation methodologies used for OCASE-T are highly varied; there is no uniform consensus. Future research should emphasize competency assessment in the operating room, validation against patient outcomes, and effectiveness for surgical training.
Collapse
Affiliation(s)
- S Swaroop Vedula
- Malone Center for Engineering in Healthcare, Department of Computer Science, The Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland 21218;
| | - Masaru Ishii
- Department of Otolaryngology-Head and Neck Surgery, The Johns Hopkins University School of Medicine, Baltimore, Maryland 21287
| | - Gregory D Hager
- Malone Center for Engineering in Healthcare, Department of Computer Science, The Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland 21218;
| |
Collapse
|
31
|
Impact of continuous training through distributed practice for acquisition of minimally invasive surgical skills. Surg Endosc 2017; 31:4051-4057. [PMID: 28236015 DOI: 10.1007/s00464-017-5451-y] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2016] [Accepted: 02/03/2017] [Indexed: 10/20/2022]
Abstract
BACKGROUND Minimally invasive surgery (MIS) requires the mastery of manual skills and a specific training is required. Apart from residencies and fellowships in MIS, other learning opportunities utilize massive training, mainly with use of simulators in short courses. A long-term postgraduate course represents an opportunity to learn through training using distributed practice. OBJECTIVE The objective of this study is to assess the use of distributed practice for acquisition of basic minimally invasive skills in surgeons who participated in a long-term MIS postgraduate course. METHODS A prospective, longitudinal and quantitative study was conducted among surgeons who attended a 1-year postgraduate course of MIS in Brazil, from 2012 to 2014. They were tested through five different exercises in box trainers (peg-transfer, passing, cutting, intracorporeal knot, and suture) in the first (t0), fourth (t1) and last, eighth, (t2) meetings of this course. The time and penalties of each exercise were collected for each participant. Participant skills were assessed based on time and accuracy on a previously tested score. RESULTS Fifty-seven surgeons (participants) from three consecutive groups participated in this study. There was a significant improvement in scores in all exercises. The average increase in scores between t0 and t2 was 88% for peg-transfer, 174% for passing, 149% for cutting, 130% for intracorporeal knot, and 120% for suture (p < 0.001 for all exercises). CONCLUSION Learning through distributed practice is effective and should be integrated into a MIS postgraduate course curriculum for acquisition of core skills.
Collapse
|
32
|
Abstract
OBJECTIVES Percutaneous renal access (PCA) is a challenging step during percutaneous nephrolithotomy. The aim of this study is to review the literature for different types of simulators described for PCA. METHODS Databases of Medline, Embase, Cochrane Library, OvidSP, and Google Scholar were systematically searched until May 2016. The studies were analyzed regarding the type of simulator (nonbiologic, biologic, live animal, and virtual reality [VR]), type of validity (face, content, construct, and predictive), cost-effectiveness, and whether these simulators have been used for training and/or assessment of PCA. In addition, the study looked at the educational impact of these simulators in terms of the transfer of PCA skills to the operating room. RESULTS Several bench, animal, and VR simulators for training in PCA were identified. Only few studies were found on assessment of PCA skills. Biological bench models used porcine or bovine kidneys wrapped within foam, silicone, chicken carcass, or full-thickness skin flap alone. Other biological models used additional subcutaneous fascia, muscle, or ribs. Nonbiological models used prototypes, including 3D printing. Only one study reported the use of anesthetized live pig for training. The PERC Mentor™ was the only VR simulator, which has been validated for training and assessment of PCA skills. However, none of these studies assessed the educational impact of PCA simulators. Furthermore, most of the studies did not address the validity and the cost of the simulator. CONCLUSIONS While several biological and nonbiological PCA models exist, there is paucity of literature regarding the validity and educational impact of these simulators. The PERC Mentor simulator is the sole validated simulator for training and assessment of PCA skills. However, it is expensive and there is little evidence of its educational impact. Therefore, more research is needed to validate the available simulators and assess their educational impact for urology trainees.
Collapse
Affiliation(s)
- Yasser A Noureldin
- 1 Division of Urology, McGill University , Montréal, Québec, Canada .,2 Urology Department, Benha University Hospital, Benha University , Benha, Egypt
| | - Sero Andonian
- 1 Division of Urology, McGill University , Montréal, Québec, Canada
| |
Collapse
|
33
|
Strategies for increasing the feasibility of performance assessments during competency-based education: Subjective and objective evaluations correlate in the operating room. Am J Surg 2016; 214:365-372. [PMID: 27634423 DOI: 10.1016/j.amjsurg.2016.07.017] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2016] [Revised: 06/23/2016] [Accepted: 07/06/2016] [Indexed: 11/22/2022]
Abstract
BACKGROUND Competency-based education necessitates assessments that determine whether trainees have acquired specific competencies. The evidence on the ability of internal raters (staff surgeons) to provide accurate assessments is mixed; however, this has not yet been directly explored in the operating room. This study's objective is to compare the ratings given by internal raters vs an expert external rater (independent to the training process) in the operating room. METHODS Raters assessed general surgery residents during a laparoscopic cholecystectomy for their technical and nontechnical performance. RESULTS Fifteen cases were observed. There was a moderately positive correlation (rs = .618, P = .014) for technical performance and a strong positive correlation (rs = .731, P = .002) for nontechnical performance. The internal raters were less stringent for technical (mean rank 3.33 vs 8.64, P = .007) and nontechnical (mean rank 3.83 vs 8.50, P = .01) performances. CONCLUSIONS This study provides evidence to help operationalize competency-based assessments.
Collapse
|
34
|
|
35
|
Lee G, Lee T, Dexter D, Klein R, Park A. Methodological Infrastructure in Surgical Ergonomics: A Review of Tasks, Models, and Measurement Systems. Surg Innov 2016; 14:153-67. [DOI: 10.1177/1553350607307956] [Citation(s) in RCA: 40] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Though in its infancy, the discipline of surgical ergonomics is increasingly valued. Still, little has been written regarding this field's tasks, models, and measurement systems. These 3 critical experimental components are crucial in objectively and accurately assessing joint and postural control as exhibited by expert laparoscopic surgeons. Such assessments will establish characteristic patterns important for surgical training. In addition, risk factors associated with both minimally invasive surgical instruments and the operating room environment can be identified and minimized. Our review focuses on evidence-based experimental ergonomic studies undertaken in the field of laparoscopic surgery. Publications were located through PubMed and other database and library searches. This article describes tasks, models, and measurement systems and considers their specific applications and the types of data obtainable with the use of each. Advantages and limitations, especially those of measurement systems, are compared and discussed. Future trends and directions believed necessary for optimal investigation and results are also addressed.
Collapse
Affiliation(s)
- Gyusung Lee
- Department of Surgery, University of Maryland, Baltimore
| | - Tommy Lee
- Department of Surgery, University of Maryland, Baltimore
| | - David Dexter
- Department of Surgery, University of Maryland, Baltimore
| | - Rosemary Klein
- Department of Surgery, University of Maryland, Baltimore
| | - Adrian Park
- Department of Surgery, University of Maryland, Baltimore,
| |
Collapse
|
36
|
Abstract
Surgical skills and simulation centers have been developed in recent years to meet the educational needs of practicing surgeons, residents, and students. The rapid pace of innovation in surgical procedures and technology, as well as the overarching desire to enhance patient safety, have driven the development of simulation technology and new paradigms for surgical education. McGill University has implemented an innovative approach to surgical education in the field of minimally invasive surgery. The goal is to measure surgical performance in the operating room using practical, reliable, and valid metrics, which allow the educational needs of the learner to be established and enable feedback and performance to be tracked over time. The GOALS system and the MISTELS program have been developed to measure operative performance and minimally invasive surgical technical skills in the inanimate skills lab, respectively. The MISTELS laparoscopic simulationtraining program has been incorporated as the manual skills education and evaluation component of the Fundamentals of Laparoscopic Surgery program distributed by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and the American College of Surgeons.
Collapse
Affiliation(s)
- Gerald M Fried
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre Hospitals, Montréal, Quebec, Canada.
| |
Collapse
|
37
|
Sant’Ana GM, Cavalini W, Negrello B, Bonin EA, Dimbarre D, Claus C, Loureiro MP, Salvalaggio PR. Retention of laparoscopic skills in naive medical students who underwent short training. Surg Endosc 2016; 31:937-944. [DOI: 10.1007/s00464-016-5063-y] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2016] [Accepted: 06/17/2016] [Indexed: 11/24/2022]
|
38
|
Zirkle M, Taplin MA, Anthony R, Dubrowski A. Objective Assessment of Temporal Bone Drilling Skills. Ann Otol Rhinol Laryngol 2016; 116:793-8. [DOI: 10.1177/000348940711601101] [Citation(s) in RCA: 50] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Objectives: There is great interest in training surgeons in the technical aspects of their craft through simulation and laboratory-based exercises. However, there are as yet only a few objective tools to assess technical performance in a laboratory setting. This study assesses three potential objective assessment tools for a traditional otolaryngology laboratory exercise, temporal bone drilling. Methods: We performed a validation study in an academic training program. Nineteen otolaryngology residents performed a cortical mastoidectomy on a cadaveric temporal bone. The participants were divided into two groups, experienced and novice, based on previous temporal bone drilling experience. Performance was rated by two independent, blinded experts using four different assessments, the Global Rating Scale (GRS), the Task-Based Checklist (TBC), the final product analysis (FPA), and expert opinion (EO). Results: The interrater reliability for all four assessments was good. Two potential objective assessments, the GRS and the TBC, and the traditional assessment tool of EO, correlated with trainee experience. The FPA, however, did not correlate with trainee experience. A logistic regression analysis of all assessments showed that the TBC correlates with EO. Conclusions: This study validates EO, the GRS, and the TBC as measures of temporal bone drilling performance. Of these measures, the TBC correlates best with EO according to logistic regression and can be reliably used as an objective assessment of temporal bone drilling.
Collapse
|
39
|
Fransson BA, Chen CY, Noyes JA, Ragle CA. Instrument Motion Metrics for Laparoscopic Skills Assessment in Virtual Reality and Augmented Reality. Vet Surg 2016; 45:O5-O13. [DOI: 10.1111/vsu.12483] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/25/2016] [Indexed: 12/14/2022]
Affiliation(s)
- Boel A. Fransson
- Department of Veterinary Clinical Sciences; Washington State University; Pullman Washington
| | - Chi-Ya Chen
- Department of Veterinary Clinical Sciences; Washington State University; Pullman Washington
| | - Julie A. Noyes
- Department of Veterinary Clinical Sciences; Washington State University; Pullman Washington
| | - Claude A. Ragle
- Department of Veterinary Clinical Sciences; Washington State University; Pullman Washington
| |
Collapse
|
40
|
Louridas M, Szasz P, de Montbrun S, Harris KA, Grantcharov TP. International assessment practices along the continuum of surgical training. Am J Surg 2016; 212:354-60. [PMID: 27018078 DOI: 10.1016/j.amjsurg.2015.12.017] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2015] [Revised: 10/25/2015] [Accepted: 12/02/2015] [Indexed: 11/15/2022]
Abstract
BACKGROUND The objectives of this study were to assemble an international perspective on (1) current, and (2) ideal technical performance assessment methods, and (3) barriers to their adoption during: selection, in-training, and certification. METHODS A questionnaire was distributed to international educational directorates. RESULTS Eight of 10 jurisdictions responded. Currently, aptitude tests or simulated tasks are used during selection, observational rating scales during training and nothing is used at certification. Ideally, innate ability should be determined during selection, in-training evaluation reports, and global rating scales used during training, whereas global and procedure-specific rating scales used at the time of certification. Barriers include lack of predictive evidence for use in selection, financial limitations during training, and a combination with respect to certification. CONCLUSIONS Identifying current and ideal evaluation methods will prove beneficial to ensure the best assessments of technical performance are chosen for each training time point.
Collapse
Affiliation(s)
- Marisa Louridas
- Department of Surgery, University of Toronto, 30 Bond Street, 16CC-056, Toronto, ON, M5B 1W8, Canada.
| | - Peter Szasz
- Department of Surgery, University of Toronto, 30 Bond Street, 16CC-056, Toronto, ON, M5B 1W8, Canada
| | - Sandra de Montbrun
- Department of Surgery, University of Toronto, 30 Bond Street, 16CC-056, Toronto, ON, M5B 1W8, Canada
| | - Kenneth A Harris
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Teodor P Grantcharov
- Department of Surgery, University of Toronto, 30 Bond Street, 16CC-056, Toronto, ON, M5B 1W8, Canada
| |
Collapse
|
41
|
Tapia-Araya AE, Usón-Gargallo J, Enciso S, Pérez-Duarte FJ, Díaz-Güemes Martin-Portugués I, Fresno-Bermejo L, Sánchez-Margallo FM. Assessment of Laparoscopic Skills in Veterinarians Using a Canine Laparoscopic Simulator. JOURNAL OF VETERINARY MEDICAL EDUCATION 2015; 43:71-79. [PMID: 26653288 DOI: 10.3138/jvme.0315-034r1] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
The aim of the present study was to assess the content and construct validity of the Canine Laparoscopic Simulator (CLS). Forty-two veterinarians were assigned to experienced (n=12), control (n=15), and training (n=15) groups, which were assessed while performing four laparoscopic tasks on the CLS. The initial and final assessments of all tasks were performed blindly by two experienced surgeons using the Global Operative Assessment of Laparoscopic Skills (GOALS) and a task-specific checklist. At the end of the study, the subjects completed an anonymous survey. The experienced group performed all of the tasks faster, with higher GOALS and checklist scores than the training and control groups (p≤.001). In the second assessment, the training group reduced the time needed to complete all of the tasks and obtained significantly higher GOALS and checklist scores than the control group. The participants perceived the CLS and its training program to be positive or very positive. The CLS and its training program demonstrated content and construct validity, supporting the suitability of the simulator for training and teaching and its ability to distinguish the degree of experience in laparoscopic surgery among veterinarians. In addition, face validity showed that the veterinarians fully accepted the CLS's usefulness for learning basic laparoscopic skills.
Collapse
|
42
|
Schwartz J, Costescu A, Mascarella MA, Young ME, Husein M, Agrawal S, Roth K, Doyle PC, Nguyen LHP. Objective assessment of Myringotomy and tympanostomy tube insertion: A prospective single-blinded validation study. Laryngoscope 2015; 126:2140-6. [DOI: 10.1002/lary.25746] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2015] [Revised: 09/21/2015] [Accepted: 09/24/2015] [Indexed: 11/08/2022]
Affiliation(s)
- Joseph Schwartz
- Department of Otolaryngology-Head and Neck Surgery; McGill University; Montreal Quebec Canada
| | - Adrian Costescu
- Faculty of Medicine; McGill University; Montreal Quebec Canada
| | - Marco A. Mascarella
- Department of Otolaryngology-Head and Neck Surgery; McGill University; Montreal Quebec Canada
| | - Meredith E. Young
- Center for Medical Education; McGill University; Montreal Quebec Canada
- Department of Medicine; McGill University; Montreal Quebec Canada
| | - Murad Husein
- Department of Otolaryngology-Head and Neck Surgery; Western University; London Ontario Canada
| | - Sumit Agrawal
- Department of Otolaryngology-Head and Neck Surgery; Western University; London Ontario Canada
| | - Kathryn Roth
- Department of Otolaryngology-Head and Neck Surgery; Western University; London Ontario Canada
| | - Philip C. Doyle
- Department of Otolaryngology-Head and Neck Surgery; Western University; London Ontario Canada
| | - Lily H. P. Nguyen
- Department of Otolaryngology-Head and Neck Surgery; McGill University; Montreal Quebec Canada
- Center for Medical Education; McGill University; Montreal Quebec Canada
| |
Collapse
|
43
|
Is there a place for virtual reality simulators in assessment of competency in percutaneous renal access? World J Urol 2015; 34:733-9. [DOI: 10.1007/s00345-015-1652-y] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2015] [Accepted: 07/22/2015] [Indexed: 10/23/2022] Open
|
44
|
A systematic review of the effect of distraction on surgeon performance: directions for operating room policy and surgical training. Surg Endosc 2015; 30:1713-24. [PMID: 26194261 DOI: 10.1007/s00464-015-4443-z] [Citation(s) in RCA: 86] [Impact Index Per Article: 8.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2015] [Accepted: 07/13/2015] [Indexed: 02/03/2023]
Abstract
BACKGROUND Distractions during surgical procedures have been linked to medical error and team inefficiency. This systematic review identifies the most common and most significant forms of distraction in order to devise guidelines for mitigating the effects of distractions in the OR. METHODS In January 2015, a PubMed and Google Scholar search yielded 963 articles, of which 17 (2 %) either directly observed the occurrence of distractions in operating rooms or conducted a laboratory experiment to determine the effect of distraction on surgical performance. RESULTS Observational studies indicated that movement and case-irrelevant conversation were the most frequently occurring distractions, but equipment and procedural distractions were the most severe. Laboratory studies indicated that (1) auditory and mental distractions can significantly impact surgical performance, but visual distractions do not incur the same level of effects; (2) task difficulty has an interaction effect with distractions; and (3) inexperienced subjects reduce their speed when faced with distractions, while experienced subjects did not. CONCLUSION This systematic review suggests that operating room protocols should ensure that distractions from intermittent auditory and mental distractions are significantly reduced. In addition, surgical residents would benefit from training for intermittent auditory and mental distractions in order to develop automaticity and high skill performance during distractions, particularly during more difficult surgical tasks. It is unclear as to whether training should be done in the presence of distractions or distractions should only be used for post-training testing of levels of automaticity.
Collapse
|
45
|
Gifford ED, Nguyen VT, Kim JJ, Schwartz SI, Chisum P, Kaji AH, Kim DY, de Virgilio C. Variation in the learning curves of general surgery residents performing arteriovenous fistulas. JOURNAL OF SURGICAL EDUCATION 2015; 72:761-766. [PMID: 25899577 DOI: 10.1016/j.jsurg.2015.02.001] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/01/2015] [Revised: 01/30/2015] [Accepted: 02/05/2015] [Indexed: 06/04/2023]
Abstract
BACKGROUND An arteriovenous fistula (AVF), performed for hemodialysis access, provides one of the few remaining opportunities for general surgery residents to perform an open vascular anastomosis (VA). Limited data exist regarding the learning curve of residents performing this procedure. The objective of this study was to determine how residents improve in performance of VA by implementing real-time tracking of anastomosis time as well as technical errors. STUDY DESIGN From April 2012 to January 2014, we conducted a prospective intraoperative assessment of 9 postgraduate year 3 general surgery residents during the performance of AVFs using a checklist of common errors in VA. Time for AVF anastomosis completion and number and types of technical errors during anastomosis were recorded. Primary end points were the change in anastomosis time and change in technical errors over time. RESULTS A total of 86 AVFs were performed and assessed intraoperatively. Each resident performed a median of 10 AVFs (interquartile range [IQR]: 7-11). The mean anastomosis time was 18.1 minutes. The mean number of technical errors was 13.8 per case. Overall, for every additional AVF performed, mean anastomosis time decreased by 0.63 minutes (95% CI: 0.45-0.81, p < 0.0001) and the mean number of technical errors decreased by 1.0 (95% CI: 0.7-1.3, p < 0.0001). The greatest improvement in overall errors (mean difference = 7.9, p = 0.03) and time (mean difference = 4.7min, p = 0.03) occurred after the performance of 3 AVFs. However, when analyzed by individual resident, the R(2) value for anastomotic time by number of AVFs performed ranged from 0.01 to 0.69. Similarly, for technical errors, the R(2) value by number of AVFs performed ranged from 0.04 to 0.62. CONCLUSIONS In novice surgical residents performing AVFs, improvement in VA skill can readily be tracked via anastomosis time and technical errors. Collectively, there is a strong association between number of cases performed and reduction in time and errors. However, individually, the number of cases completed did not correlate well with time and errors. These findings suggest that for VA skills, determining progression from novice to competence cannot rely on case volume but rather needs to be individualized.
Collapse
Affiliation(s)
- Edward D Gifford
- Department of Surgery, Harbor-UCLA Medical Center, Torrance, California
| | - Virginia T Nguyen
- Department of Surgery, Harbor-UCLA Medical Center, Torrance, California
| | - Jerry J Kim
- Department of Surgery, Harbor-UCLA Medical Center, Torrance, California
| | - Samuel I Schwartz
- Department of Surgery, Harbor-UCLA Medical Center, Torrance, California
| | - Patrick Chisum
- Department of Surgery, Harbor-UCLA Medical Center, Torrance, California
| | - Amy H Kaji
- Harbor-UCLA Medical Center, Torrance, California
| | - Dennis Y Kim
- Department of Surgery, Harbor-UCLA Medical Center, Torrance, California
| | | |
Collapse
|
46
|
Zendejas B, Ruparel RK, Cook DA. Validity evidence for the Fundamentals of Laparoscopic Surgery (FLS) program as an assessment tool: a systematic review. Surg Endosc 2015; 30:512-520. [PMID: 26091982 DOI: 10.1007/s00464-015-4233-7] [Citation(s) in RCA: 82] [Impact Index Per Article: 8.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2014] [Accepted: 05/08/2015] [Indexed: 10/23/2022]
Abstract
BACKGROUND The Fundamentals of Laparoscopic Surgery (FLS) program uses five simulation stations (peg transfer, precision cutting, loop ligation, and suturing with extracorporeal and intracorporeal knot tying) to teach and assess laparoscopic surgery skills. We sought to summarize evidence regarding the validity of scores from the FLS assessment. METHODS We systematically searched for studies evaluating the FLS as an assessment tool (last search update February 26, 2013). We classified validity evidence using the currently standard validity framework (content, response process, internal structure, relations with other variables, and consequences). RESULTS From a pool of 11,628 studies, we identified 23 studies reporting validity evidence for FLS scores. Studies involved residents (n = 19), practicing physicians (n = 17), and medical students (n = 8), in specialties of general (n = 17), gynecologic (n = 4), urologic (n = 1), and veterinary (n = 1) surgery. Evidence was most common in the form of relations with other variables (n = 22, most often expert-novice differences). Only three studies reported internal structure evidence (inter-rater or inter-station reliability), two studies reported content evidence (i.e., derivation of assessment elements), and three studies reported consequences evidence (definition of pass/fail thresholds). Evidence nearly always supported the validity of FLS total scores. However, the loop ligation task lacks discriminatory ability. CONCLUSION Validity evidence confirms expected relations with other variables and acceptable inter-rater reliability, but other validity evidence is sparse. Given the high-stakes use of this assessment (required for board eligibility), we suggest that more validity evidence is required, especially to support its content (selection of tasks and scoring rubric) and the consequences (favorable and unfavorable impact) of assessment.
Collapse
Affiliation(s)
- Benjamin Zendejas
- Department of Surgery, Mayo Clinic College of Medicine, Mayo 12-W, 200 First Street SW, Rochester, MN, 55905, USA.
| | - Raaj K Ruparel
- Department of Surgery, Mayo Clinic College of Medicine, Mayo 12-W, 200 First Street SW, Rochester, MN, 55905, USA
| | - David A Cook
- Division of General Internal Medicine, Mayo Clinic College of Medicine, Rochester, MN, USA.,Multidisciplinary Simulation Center, Mayo Clinic College of Medicine, Rochester, MN, USA
| |
Collapse
|
47
|
Szasz P, Louridas M, Harris KA, Aggarwal R, Grantcharov TP. Assessing Technical Competence in Surgical Trainees. Ann Surg 2015; 261:1046-55. [PMID: 25119118 DOI: 10.1097/sla.0000000000000866] [Citation(s) in RCA: 73] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023]
|
48
|
Cavalini WLP, Claus CMP, Dimbarre D, Cury Filho AM, Bonin EA, Loureiro MDP, Salvalaggio P. Development of laparoscopic skills in medical students naive to surgical training. ACTA ACUST UNITED AC 2015; 12:467-72. [PMID: 25628198 PMCID: PMC4879913 DOI: 10.1590/s1679-45082014ao3237] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2014] [Accepted: 10/30/2014] [Indexed: 06/24/2024]
Abstract
OBJECTIVE To assess the acquisition of basic laparoscopic skills of Medical students trained on a surgical simulator. METHODS First- and second-year Medical students participated on a laparoscopic training program on simulators. None of the students had previous classes of surgical technique, exposure to surgical practice nor training prior to the enrollment in to the study. Students´ time were collected before and after the 150-minute training. Skill acquisition was measured comparing time and scores of students and senior instructors of laparoscopic surgery. RESULTS Sixty-eight students participated of the study, with a mean age of 20.4 years, with a predominance of first-year students (62%). All students improved performance in score and time, after training (p<0,001). Score improvement in the exercises ranged from 294.1 to 823%. Univariate and multivariate analyses identified that second-year Medical students have achieved higher performance after training. CONCLUSIONS Medical students who had never been exposed to surgical techniques can acquire basic laparoscopic skills after training in simulators. Second-year undergraduates had better performance than first-year students.
Collapse
|
49
|
Olasky J, Sankaranarayanan G, Seymour NE, Magee JH, Enquobahrie A, Lin MC, Aggarwal R, Brunt LM, Schwaitzberg SD, Cao CGL, De S, Jones DB. Identifying Opportunities for Virtual Reality Simulation in Surgical Education: A Review of the Proceedings from the Innovation, Design, and Emerging Alliances in Surgery (IDEAS) Conference: VR Surgery. Surg Innov 2015; 22:514-21. [PMID: 25925424 DOI: 10.1177/1553350615583559] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
Abstract
OBJECTIVES To conduct a review of the state of virtual reality (VR) simulation technology, to identify areas of surgical education that have the greatest potential to benefit from it, and to identify challenges to implementation. BACKGROUND DATA Simulation is an increasingly important part of surgical training. VR is a developing platform for using simulation to teach technical skills, behavioral skills, and entire procedures to trainees and practicing surgeons worldwide. Questions exist regarding the science behind the technology and most effective usage of VR simulation. A symposium was held to address these issues. METHODS Engineers, educators, and surgeons held a conference in November 2013 both to review the background science behind simulation technology and to create guidelines for its use in teaching and credentialing trainees and surgeons in practice. RESULTS Several technologic challenges were identified that must be overcome in order for VR simulation to be useful in surgery. Specific areas of student, resident, and practicing surgeon training and testing that would likely benefit from VR were identified: technical skills, team training and decision-making skills, and patient safety, such as in use of electrosurgical equipment. CONCLUSIONS VR simulation has the potential to become an essential piece of surgical education curriculum but depends heavily on the establishment of an agreed upon set of goals. Researchers and clinicians must collaborate to allocate funding toward projects that help achieve these goals. The recommendations outlined here should guide further study and implementation of VR simulation.
Collapse
Affiliation(s)
- Jaisa Olasky
- Mount Auburn Hospital, Harvard Medical School, Cambridge, MA, USA
| | | | - Neal E Seymour
- Tufts University School of Medicine, Springfield, MA, USA
| | - J Harvey Magee
- University of Maryland Medical Center, Baltimore, MD, USA
| | | | - Ming C Lin
- The University of North Carolina at Chapel Hill, NC, USA
| | - Rajesh Aggarwal
- University of Pennsylvania Medical School, Philadelphia, PA, USA
| | | | | | | | - Suvranu De
- Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Daniel B Jones
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
50
|
Jirapinyo P, Kumar N, Thompson CC. Validation of an endoscopic part-task training box as a skill assessment tool. Gastrointest Endosc 2015; 81:967-73. [PMID: 25310934 DOI: 10.1016/j.gie.2014.08.007] [Citation(s) in RCA: 29] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/08/2014] [Accepted: 08/06/2014] [Indexed: 12/13/2022]
Abstract
BACKGROUND There is no objective methodology to assess trainee progress in endoscopy. Our prior work has detailed the development of the endoscopic part-task training box. OBJECTIVE To assess validity evidence regarding relationship to other variables by evaluating a correlation between level of endoscopic experience and training box score. DESIGN Prospective validation study. SETTING Three academic institutions. PARTICIPANTS A total of 42 participants: 7 novices, 7 first-year GI fellows, 7 second-year GI fellows, 7 third-year GI fellows, 7 attending physicians, and 7 interventional attending physicians. INTERVENTIONS The training box consists of 5 modules: retroflexion, knob control, torque, polypectomy, and navigation/loop reduction. Performance is scored for precision and speed. Each participant was required to complete the training box once. Additionally, 5 participants at different endoscopic levels completed the training box 3 times at 1-week intervals. MAIN OUTCOME MEASUREMENTS A correlation between level of endoscopic experience and training box score. RESULTS All 42 participants completed the 5 modules during a single session. Aggregate training box scores differed significantly between each training level (P values < .05). Individual modules significantly differentiated between experience-level groups (novices, fellows, and attending physicians; P values < .01). Participants who repeated the training box demonstrated score improvement over time, with persistence of separation between training levels. LIMITATIONS The training box focuses only on the technical aspects of endoscopy and does not address the cognitive elements of endoscopic training. CONCLUSION The endoscopic part-task training box is able to objectively assess endoscopic ability by differentiating scores based on clinical experience. Further multicenter efforts are now needed to establish learning curves and to correlate use of the simulator with improved clinical aptitude.
Collapse
Affiliation(s)
- Pichamol Jirapinyo
- Division of Gastroenterology, Brigham and Women's Hospital, Boston, Massachusetts, USA
| | - Nitin Kumar
- Division of Gastroenterology, Brigham and Women's Hospital, Boston, Massachusetts, USA
| | | |
Collapse
|