Prospective Study Open Access
Copyright ©The Author(s) 2020. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastrointest Endosc. Mar 16, 2020; 12(3): 98-110
Published online Mar 16, 2020. doi: 10.4253/wjge.v12.i3.98
Impact of a simulation-based induction programme in gastroscopy on trainee outcomes and learning curves
Keith Siau, James Hodson, Peter Neville, Jeff Turner, Amanda Beale, Susi Green, Aravinth Murugananthan, Paul Dunckley, Neil D Hawkes
Keith Siau, Paul Dunckley, Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London NW1 4LE, United Kingdom
Keith Siau, Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, United Kingdom
Keith Siau, James Hodson, Institute of Translational Medicine, University Hospitals Birmingham, Birmingham B15 2TH, United Kingdom
Peter Neville, Neil D Hawkes, Department of Gastroenterology, Cwm Taf Morgannwg Health Board, Llantrisant CF45 4SN, United Kingdom
Jeff Turner, Department of Gastroenterology, Cardiff and Vale University Health Board, Cardiff CF14 4XW, United Kingdom
Amanda Beale, Department of Gastroenterology, University Hospitals Bristol NHSFT, Bristol BS1 3NU, United Kingdom
Susi Green, Department of Gastroenterology, Royal Sussex County Hospital, Brighton BN2 5BE, United Kingdom
Aravinth Murugananthan, Department of Gastroenterology, Royal Wolverhampton NHS Trust, Wolverhampton W10 0QP, United Kingdom
Paul Dunckley, Department of Gastroenterology, Gloucestershire Hospitals NHSFT, Gloucester GL1 3NN, United Kingdom
ORCID number: Keith Siau (0000-0002-1273-9561).
Author contributions: Siau K contributed to the study design, statistical analyses, literature review, initial draft of the manuscript; Hodson J contributed to the statistical analyses, critical review of manuscript; Neville P, Turner J, Beale A, Green S, Murugananthan A contributed to the study design, conduct of study, training faculty, critical review of manuscript; Dunckley P contributed to the study design, critical review of manuscript; Hawkes ND contributed to the study conception, oversight and guarantorship.
Institutional review board statement: Study approval was granted by JAG Quality Assurance of Training working group.
Clinical trial registration statement: This was not a clinical trial, not register.
Informed consent statement: All participants provided written, informed consent for inclusion within the study.
Conflict-of-interest statement: None of the authors have any conflicts of interest to declare. Surgical Science was blinded to the results of the study.
Data sharing statement: There is no additional data available.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Corresponding author: Keith Siau, MBChB, MRCP, Doctor, Senior Research Fellow, Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, 11 St Andrew's Place, London NW1 4LE, United Kingdom. keithsiau@nhs.net
Received: October 24, 2019
Peer-review started: October 24, 2019
First decision: November 20, 2019
Revised: December 21, 2019
Accepted: February 23, 2020
Article in press: February 23, 2020
Published online: March 16, 2020

Abstract
BACKGROUND

Pre-clinical simulation-based training (SBT) in endoscopy has been shown to augment trainee performance in the short-term, but longer-term data are lacking.

AIM

To assess the impact of a two-day gastroscopy induction course combining theory and SBT (Structured PRogramme of INduction and Training – SPRINT) on trainee outcomes over a 16-mo period.

METHODS

This prospective case-control study compared outcomes between novice SPRINT attendees and controls matched from a United Kingdom training database. Study outcomes comprised: (1) Unassisted D2 intubation rates; (2) Procedural discomfort scores; (3) Sedation practice; (4) Time to 200 procedures; and (5) Time to certification.

RESULTS

Total 15 cases and 24 controls were included, with mean procedure counts of 10 and 3 (P = 0.739) pre-SPRINT. Post-SPRINT, no significant differences between the groups were detected in long-term D2 intubation rates (P = 0.332) or discomfort scores (P = 0.090). However, the cases had a significantly higher rate of unsedated procedures than controls post-SPRINT (58% vs 44%, P = 0.018), which was maintained over the subsequent 200 procedures. Cases tended to perform procedures at a greater frequency than controls in the post-SPRINT period (median: 16.2 vs 13.8 per mo, P = 0.051), resulting in a significantly greater proportion of cases achieving gastroscopy certification by the end of follow up (75% vs 36%, P = 0.017).

CONCLUSION

In this pilot study, attendees of the SPRINT cohort tended to perform more procedures and achieved gastroscopy certification earlier than controls. These data support the role for wider evaluation of pre-clinical induction involving SBT.

Key Words: Gastroscopy, Esophagogastroduodenoscopy, Endoscopy training, Induction, Competency development, Simulation

Core tip: Simulation-based training has been shown to improve short-term trainee outcomes, but longer-term data on trainee and patient-based outcomes are lacking. A 2-d induction programme covering fundamental theory and hands-on training can improve trainee confidence and shorten the time to achieve gastroscopy certification.



INTRODUCTION

High quality training is a prelude to high quality endoscopy[1]. Within the United Kingdom (UK), quality assurance of endoscopy training is overseen by the Joint Advisory Group on Gastrointestinal Endoscopy (JAG)[2]. For most gastroenterologists, training in endoscopy begins with gastroscopy. The process of gastroscopy training requires considerable time and effort; on average, 187 procedures are required to achieve consistent gastroscopy completion rates (95%+ intubation to the second part of the duodenum – D2)[3] and 282 procedures (1.9 years) to attain JAG certification[4], which is a national requirement for independent practice. With the imminent “Shape of Training” reforms to UK gastroenterology training[5], which proposes to shorten the length of specialist training, endoscopy trainers need to re-evaluate training methods and tools, to deliver evidence-based training pathways which accelerate the development of competency in endoscopic procedures.

Simulation-based training (SBT) provides one solution to this challenge. Modern-day computerised virtual reality (VR) simulators are capable of delivering immersive training without risk of patient harm. The plethora of high-quality evidence attests to the short-term benefits of SBT in augmenting the acquisition of fundamental technical skills such as scope handling and tip control[6-8], which could shorten the learning curve. Additionally, data from colonoscopy training confirm that trainee endoscopists incur more procedural discomfort during earlier stages of training[9]. Despite this, pre-clinical SBT is not readily available within the UK JAG training pathway, where hands-on training typically begins with patient-based gastroscopy at the discretion of supervising trainers. Hence, there is a need for a standardised induction programme which can ensure that beginners are sufficiently armed with the basic skills and knowledge before approaching patient-based training, in line with other international training pathways[10]. Data on the longer-term benefits of SBT on trainee and patient outcomes are lacking.

The Structured PRogramme of INduction and Training (SPRINT) is an induction programme consisting of a structured sequence of theory and hands-on training elements designed to optimise and accelerate the early phase of training in gastroscopy (Table 1). The didactic theory-based seminars are intended to complement SBT and cover fundamental aspects such as endoscope design/handling and basic lesion recognition. The EndoSim (Surgical Science, Gothenburg, Sweden) is a novel endoscopic VR simulator (Figure 1) which incorporates a customisable SBT curriculum and generates task-specific metrics, but has not been validated. In September 2017, a two-day gastroscopy Induction programme combining these foundation knowledge sessions with SBT was provided to medical and surgical trainees from three training deaneries. The primary aim of this training intervention was to assess whether this type of enhanced induction can accelerate competency development in novice trainees. The secondary aim was to evaluate whether EndoSim metrics could distinguish between trainees and experts in order to assess discriminative validity of the simulator.

Table 1 The Structured PRogramme of INduction and Training gastroscopy induction programme.
TimeProgramme
8.3Coffee and registration
9Welcome and introduction to aims and objectives
9.3Simulator session 1Basic handling and scope design
10.2Basic handling and scope designSimulator session 1
11.1Coffee
11.3Simulator session 2JAG Certification, appraisal and training lists
12.2JAG Certification, appraisal and training listsSimulator session 2
13.1Lunch
13.4Simulator session 3Enhancing the endoscopic image
14.1Enhancing the endoscopic imageSimulator session 3
15Coffee
15.2Simulator session 4Lesion recognition and assessment skills 1
16.1Lesion recognition and assessment skills 1Simulator session 4
17Round up
8.3Coffee and registration
9Welcome and introduction to day 2
9.1Simulator session 5Getting the best out of the JETS e-portfolio
10Getting the best out of the JETS e-portfolioSimulator session 5
10.5Coffee
11.1Simulator session 6Lesion recognition and assessment skills 2
12Lesion recognition and assessment skills 2Simulator session 2
12.5Lunch
13.2Simulator session 7Decision-making and report writing
13.5Decision-making and report writingSimulator session 7
14.4Coffee
15Simulator session 8DOPS assessment and improving your skills
15.5DOPS assessment and improving your skillsSimulator session 8
16.4Summary and review of course objectives
Figure 1
Figure 1 The surgical science EndoSim simulation module. A: Endoscopy stack; B: Virtual reality views of the duodenum; C: Gastric cardia; D: Endoscope configuration.
MATERIALS AND METHODS
Study design

In this prospective multicentre study, trainees commencing gastroscopy training (ST3) from three UK training deaneries (regions) were enrolled to the augmented SBT induction programme. All trainees completed a structured curriculum of hands-on simulator training lasting a minimum of 3 h with feedback from JAG certified faculty trainers. This study had three components. First, trainee performance on the EndoSim VR simulator was compared to that of the expert faculty (all > 1000 procedures) to explore discriminative validity of the metrics used. To minimise bias, each participant’s first valid attempt, without prior faculty training or feedback, was included. Training was only offered once all trainees had completed the assessment round. The second component was an assessment of the change in self-reported confidence scores following the course, based on questionnaires completed immediately pre- and post-course. The third component of the study assessed the impact of the course on long-term trainee outcomes using a case-control method. In the UK, all trainees are required to log training procedures onto the JAG Endoscopy Training System (JETS) e-portfolio[11]. Participation is mandated to enable certification for independent practice. For this analysis, a cohort of control trainees was selected from non-attenders who had submitted formative assessment data on JETS in the post-course period between September and December 2017. Only trainees with < 50 procedures, and whose first JETS-recorded procedure was less than a year prior to the date of SPRINT were included in the analyses of long-term outcomes, to ensure the levels of experience were similar in cases and controls. In addition, trainees with no gastroscopy procedures logged on JETS post-SPRINT were excluded, since these had not begun hands-on training in the post-course period. Trainee and patient outcomes for each post-course training procedure were extracted from the JETS e-portfolio, with prospective follow-up of outcomes post-SPRINT performed from September 2017 until February 2019 (maximum period of 16 mo).

Study approval

Study approval was granted by JAG Quality Assurance of Training working group. All participants provided written, informed consent for inclusion within the study. Neither the researchers nor Surgical Science had access to the study outcome data over the course of the study. There was no financial incentive to conduct this study.

Study outcomes

Discriminative validity of EndoSim: EndoSim scenarios and computer-generated metrics relevant to the assessment of technical skills in gastroscopy were selected as study outcomes for the first component of the study. Pre-set modules relevant to gastroscopy were selected, which generated skillset-dependent EndoSim metrics. Comparisons were then made between trainees and experts, followed by subgroup analyses within the trainees, comparing novices (< 25 procedures) to those with intermediate experience (25 + procedures).

Impact on self-assessed competence scores: Questionnaires were administered to all trainees both pre- and post-course, which measured self-assessed competency scores in 12 upper GI handling skills domains. These domains were mapped to the formative JAG formative assessment forms which are integrated into UK endoscopy training[12]. Domain scores were given as a rating from 0-10 on a Likert scale (0 = not at all competent, 10 = very competent).

Long-term trainee outcomes: For the analysis of long-term trainee outcomes, the trainee outcomes included the unassisted rate of D2 intubation, i.e., procedures without physical assistance, the volume of training procedures performed post-course, and the time taken to achieve JAG gastroscopy certification. JAG certification involves the composite outcome of attaining a minimum lifetime patient-based gastroscopy count (200+) and satisfactory completion of formative and summative direct observation of procedural skills assessments to objectively demonstrate competence[4,12,13]. Patient outcomes were also explored; these included comparisons in rates of moderate-to-severe discomfort and rates of unsedated procedures.

Statistical analyses

For the first two components of the study, variables were reported as medians and interquartile ranges (IQRs), and compared between groups using Mann-Whitney tests, with Wilcoxon’s tests used for paired comparisons. For the third component, trends in study outcomes (i.e., D2 intubation, sedation and discomfort rates) were evaluated over the 200 post-SPRINT procedures using generalised estimating equation (GEE) models, to account for the non-independence of repeated procedures by the same trainee. Prior to the analysis, the relationship between procedural experience and outcome measures were assessed graphically, and transformations applied, as applicable, to ensure goodness-of-fit. Binary logistic GEE models were then produced, using an autoregressive correlation structure. The trainee group (case/control) and the procedure number post-SPRINT were set as covariates, with an interaction term also included in the model. As such, the analysis allowed for the two groups to have different outcome rates at baseline, but also allowed for the rate of improvement over time to vary between the groups, with the interaction term allowing for comparison of the latter.

The times taken to reach the 200th procedure and for gastroscopy certification were assessed with Kaplan-Meier plots, with comparisons between cases and controls made using univariable Cox regression models. Trainees who did not achieve these outcomes were censored at the date of their final procedure. The procedure counts in the post-SPRINT period were then compared. To account for the potential loss of follow-up after gastroscopy certification, procedures performed after certification were excluded. Since this resulted in differing durations of follow up across the trainees, the total procedure counts were then divided by the time between the SPRINT course and gastroscopy certification, or to the final procedure date in those without certification, and analysed as an average number of procedures per month. All analyses were performed using SPSS v24 (IBM Corp. Armonk, NY), with P < 0.05 indicating statistical significance.

RESULTS
Participants

In total, 20 trainees and 6 faculty members (experts) attended the SPRINT induction programme. Of these, 10 trainees were classified as novices (< 25 procedures) and 10 as having intermediate experience (25+ procedures). Data from all participants were included in the analyses of EndoSim metrics and self-assessment scores. For the learning curve analyses, trainees were excluded on the basis of: Having no procedures recorded in the JETS e-portfolio (n = 3), > 50 pre-course procedures (n = 1), and first recorded procedure > 1 year before the course (n = 1), leaving 15 cases for analysis. Data for 24 control patients were identified. The majority of cases (93%) and controls (88%) were gastroenterology trainees, with upper gastrointestinal surgical trainees comprising the remainder. There were no significant differences in trainee specialty between groups (P = 0.967). Prior to the date of SPRINT, the average number of procedures performed was similar between the two groups (P = 0.739), with a mean of 10 per trainee in cases and 3 per trainee in controls; 63% of controls and 60% of cases had performed zero procedures prior to the course date.

Discriminative validity of EndoSim metrics

Comparisons of EndoSim metrics between trainees (novice and intermediate) and experts are presented in Table 2. Five gastroscopy-relevant modules were selected comprising: wheel handling, navigation, button handling, photodocumentation and biopsy, each with a variable number of substations. All trainees and faculty successfully completed and passed each station. Trainees could be differentiated from experts for at least one metric on all modules except for the “button handling” station. For the remaining stations, experts could be delineated from trainee performance in terms of efficiency metrics (i.e., total time to complete a task), efficiency of movement (e.g., more conservative wheel and scope rotation and less endoscope tip path length in the Navigation module, fewer collisions against the mucosa) and precision (fewer missed targets and more biopsies within target). Of the 22 metrics relevant to the five modules, 5 (23%) were significantly different between novice and intermediate trainees and 14 (64%) between trainees and experts.

Table 2 Comparisons of module-dependent EndoSim metrics between trainees (stratified into novice and intermediate experience groups) and faculty members.
ModuleMetricMedian (IQR)
P value (Expert vs trainee)Median (IQR)
P value (Novice vs intermediate)
Expert (n =6)Trainee (n = 20)Novice trainee (n = 10)Intermediate trainee (n = 10)
Module 1: Wheel Handling (4 stations)Missed targets3 (1-4)6 (3-8)< 0.0017 (4-9)6 (2-8)0.057
Wheel rotation left/right (Degrees)257 (42-382)143 (5-591)0.46382 (1-643)166 (9-575)0.753
Wheel rotation up/down (Degrees)783 (691-916)764 (606-1173)0.903680 (442-1005)1023 (687-1303)0.003
Endoscope rotation (Degrees)1398 (749-2355)964 (353-1577)0.025886 (350-1404)1044 (349-1955)0.350
Module 2: Navigation (3 stations)Total time (s)74 (52-104)104 (79-166)0.002161 (108-218)82 (67-105)< 0.001
Wheel rotation left/right (Degrees)109 (40-305)138 (3-757)0.826391 (3-1648)99 (2-549)0.143
Wheel rotation up/down (Degrees)888 (680-1108)1232 (934-1868)0.0011268 (958-1737)1224 (931-2046)0.641
Endoscope rotation (Degrees)1120 (933-1865)1770 (1313-2334)0.0071847 (1258-2571)1722 (1357-2258)0.503
Endoscope tip path length (cm)228 (179-306)324 (251-411)0.002357 (280-489)280 (239-356)0.028
Module 3: Button Handling (3 stations)Missed targets (number)2 (0-4)2 (1-4)0.6232 (1-5)1.5 (1-4)0.805
Unnecessary button presses (number)2 (0-4)2 (1-4)0.2702 (1-5)2 (1-4)0.963
Missed dirt (number)1 (1-2)1 (1-2)0.9441 (0-2)1 (1-2)0.429
Module 4: Photo (4 stations)Total time (s)151 (121-192)313 (230-377)< 0.001328 (235-404)269 (179-361)0.054
Stomach visualized (%)93% (79%-99%)100 (96%-100%)< 0.00199% (94%-100%)100% (97%-100%)0.070
Duodenum visualized (%)63% (52%-74%)63% (53-72%)0.85562% (51%-68%)66% (58%-74%)0.088
Collisions against mucosa (number)8 (5-12)13 (9-16)< 0.00113 (11-20)12 (8-15)0.090
Targets photographed (%)100% (100%-100%)100% (100%-100%)0.495100% (100%-100%)100% (100%-100%)0.302
Module 5: Biopsy (3 stations)Total time (s)182 (163-217)340 (249-463)< 0.001446 (331-522)299 (215-389)0.001
Targets biopsied100% (100%-100%)100% (50-100%)0.010100% (38%-100%)100% (50%-100%)0.546
Biopsies outside any target (number)0 (0-2)4 (2-9)< 0.0013 (2-8)4 (2-11)0.548
Collisions against mucosa (number)7 (4-11)9 (7-13)0.03012 (9-23)7 (6-11)< 0.001
Movement with tool (cm)25 (17-53)72 (36-183)0.00273 (29-183)70 (42-179)0.910
Self-assessment scores

The 20 trainees attending SPRINT reported their confidence in 12 different skills both pre- and post-course. Across these skills, the median confidence score ranged from 3-5 on the pre-course questionnaire (Table 3). After the course, confidence in all 12 skills increased significantly (all P < 0.001), with medians ranging from 7-9.

Table 3 Self-reported scores pre- and post-course.
SkillMedian confidence score (IQR)
P value
Pre-coursePost-course
Tip control5 (2-7)8 (7-9)< 0.001
Torque steering5 (2-6)8 (7-9)< 0.001
Intubation3 (0-7)7 (6-9)< 0.001
Oesophagus to pylorus5 (1-8)9 (7-9)< 0.001
Pyloric intubation4 (0-7)8 (7-9)< 0.001
D2 intubation3 (0-6)7 (5-9)< 0.001
Duodenal withdrawal4 (0-7)8 (5-9)< 0.001
J manoeuvre5 (1-8)8 (7-9)< 0.001
Retroflexed views5 (2-7)8 (6-9)< 0.001
Overall visualisation5 (2-7)8 (7-9)< 0.001
Image taking4 (1-6)8 (7-8)< 0.001
Use of accessories3 (0-5)8 (6-9)< 0.001
Trends in procedural outcomes by course attendance

Preliminary analysis of the trends in unassisted D2 intubation rates found these to increase rapidly over the initial post-course procedures, with the rate of improvement slowing after 25-50 procedures and flattening subsequently as the D2 intubation rates neared 90%. Due to this rapid change in gradient, a binary logistic regression model with the lifetime procedure number as a covariate resulted in a poor fit. As such, lifetime procedure counts were log2-transformed, after adding 10, which improved the goodness-of-fit of the model.

This model was then applied to the cases and controls separately, using a GEE approach, to compare the rate of improvement between groups (Table 4, Figure 2A). This showed comparable baseline unassisted D2 intubation rates, with estimates of 28% for cases and 35% for controls at the first procedure post-SPRINT (P = 0.332). Unassisted D2 intubation rates improved with experience, with a doubling of the procedure count associated with an odds ratio of 1.99 in cases (P < 0.001) and 1.74 (P < 0.001) in controls (Table 4). However, no significant difference was detected between these gradients (P = 0.205). Hence, the rates of improvement in unassisted D2 intubation rates by procedure number were comparable between cases and controls.

Figure 2
Figure 2 Plots of outcome rates by post-course procedure number, stratified according to cases and controls. A: Unassisted D2 intubation rates; B: Rates of moderate-severe discomfort; C: Unsedated procedures. Trendlines are extrapolated from generalised estimating equation models, as described in Table 4.
Table 4 Generalised estimating equation models of procedure outcomes in cases and controls.
Cases
Controls
P value (Case vs Control)
Odds ratio (95%CI)P valueOdds ratio (95%CI)P value
Unassisted D2 intubation rates
Intercept0.51 (0.13 – 1.98)---0.332
Gradient (per Doubling of OGD count)1.99 (1.69 – 2.34)< 0.0011.74 (1.53 – 1.98)< 0.0010.205
Moderate-severe discomfort
Intercept0.42 (0.15 - 1.15)---0.09
Gradient (per 10 procedures)0.97 (0.88 - 1.07)0.5260.92 (0.85 - 1.00)0.0440.421
Unsedated procedures
Intercept1.63 (1.09 – 2.46)---0.018
Gradient (per 10 procedures)0.99 (0.97 – 1.01)0.281.00 (0.98 – 1.02)0.9730.445

Of the other outcomes considered, no significant differences in the rates of moderate-severe discomfort were detected between the case and control groups (Figure 2B). However, significant differences in the proportions of unsedated procedures were observed (Figure 2C). At baseline, the cases performed a significantly greater proportion of procedures without sedation (odds ratio: 1.63, 95%CI: 1.09–2.46, P = 0.018), with 58% of the first ten procedures by cases being unsedated, compared to 44% of those by controls. This difference between the groups was then sustained over the subsequent procedures (interaction term: P = 0.445).

Times to performance milestones

Over the post-SPRINT follow-up period of 16 mo, 13/15 (87%) cases and 15/24 (63%) controls reached a lifetime procedure count of 200. The Kaplan-Meier estimated median time to the 200th procedure (Figure 3A) did not differ significantly (P = 0.190) between cases (10.6 mo) and controls (12.1 mo). Gastroscopy certification was achieved in 11/15 (73%) cases and 7/24 controls (29%). Delegates achieved certification after a median time of 14 mo post-SPRINT, which was significantly earlier than controls (Figure 3B, P = 0.017), for whom the rate did not reach 50% (i.e., the median time was > 16 mo). By the end of follow up (i.e., 16 mo), the Kaplan-Meier estimated certification rates were 75% vs 36% in cases vs controls.

Figure 3
Figure 3 Kaplan-Meier curves of time to 200 procedures (A) and time to gastroscopy certification (B).
Post-course procedure counts

Cases performed a median of 16.2 (IQR: 13.8-20.8) procedures per month post-SPRINT, which was higher than the 13.8 (IQR: 9.2-16.7) observed in controls, although this missed statistical significance (P = 0.051).

DISCUSSION

In this small prospective case-control pilot study, trainees who attended a two-day hands-on gastroscopy induction course involving basic theory and SBT showed no significant difference in the learning curve to achieve unassisted D2 intubation. However, attenders achieved JAG certification earlier than peers from the control group, which may be explained by the tendency to perform more post-course procedures (P = 0.051). These results provide real-world data on the durability of an SBT induction programme on trainee and patient outcomes.

To account of the possibility of trainees attempting harder to complete an examination, discomfort and sedation-based outcomes were also explored. Course attenders performed more post-course procedures without sedation (P = 0.018), but without a significant difference in rates of moderate-severe discomfort (P = 0.090). To our knowledge, the only other publication which assessed the durability of SBT over 200 procedures was the randomised controlled trial (RCT) by Cohen et al[14] in the context of colonoscopy training. Trainees allocated to pre-clinical SBT demonstrated superior technical and cognitive outcomes during early stages of training, but this effect dissipated after 100 procedures. Previous RCTs on gastroscopy training have assessed post-SBT outcomes after 3-4 wk[15,16], or after 2-60 procedures of patient-based training[17-20], with the majority showing improvements in favour of SBT. Study protocols have also varied with regard to the duration of SBT exposure and training structure. Di Giulio et al[20] found that trainees randomised to 10 h of SBT performed better at unassisted D2 intubation, retroflexion and landmark identification over 20 patient-based procedures (88% vs 70%, P < 0.001). In another RCT, trainees who underwent 2 h of SBT demonstrated improved D2 intubation over 10 procedures, but no difference in patient discomfort scores[19].

Not all studies have associated SBT with improved outcomes. The Sedlack[16] study found no benefit in trainee gastroscopy outcomes after 6 h of SBT. Concerns over the face validity of the simulator was cited as a possible explanation. Our study presents novel evidence of discriminant validity of the EndoSim platform and response process validity in form of trainee feedback and the improvements in self-confidence scores. Although we found no significant difference between cases and controls in unassisted D2 intubation rates by procedure count, the rates at which trainees acquired competence, as evidenced by time-to-competency endpoints, was found in favour of SPRINT course attenders. It is recognised that coaching and feedback[8], coupled with a structured SBT curriculum[21], and a minimum exposure period to SBT[1], may be required to unlock the full potential of SBT. Notably, the difference between groups in rates of unsedated procedures may be another confounding factor.

Our study provides novel data from the perspective of UK-based endoscopy training which has its imperfections[4,22]. In addition to the lack of standardised SBT-curricula, training occurs within endoscopy units which face the perennial dilemma of balancing service capacity with list reductions to accommodate novice trainees. Trainees often face competing commitments, e.g., from on-call rotas, ward and clinic duties, and may have to compete for training with other specialties[22]. It is possible that the improvements in trainee confidence derived from SPRINT may empower trainees to train on unsedated procedures and adopt a more proactive training stance, thereby leading to greater acquisition of training experience and shorter times to certification.

Our study had several noteworthy limitations. First, this was a non-randomised pilot study. Despite the similar numbers of pre-course procedures in the two groups, we cannot exclude differences in training provisions within the training regions for cases and controls. Second, this pilot study, with its 15 eligible cases, may be underpowered to detect statistically significant differences in long-term study outcomes. No formal power calculation was performed prior to study commencement, as the included trainees consisted of a convenience sample of SPRINT course attendees, hence there was no scope to increase the sample size. Third, not all trainees were fully novices, with approximately 40% having some degree of gastroscopy experience. However, this was generally limited to a small number of procedures and did not differ significantly between cases and controls. Fourth, the validity of EndoSim was not rigorously appraised, as the primary intention was to provide training and to assess longer term outcomes. Further evaluation is required to appraise face validity, i.e., realism, and comparisons of EndoSim performance over time. Fifth, owing to the comparisons of trainee and faculty performance of the EndoSim simulator, modules were initially performed by trainees without coaching or feedback, which are pivotal for skills acquisition with SBT. Faculty experts were unfamiliar to EndoSim and did not receive pre-course training. These factors may have compromised the effectiveness of hands-on technical skills training. Sixth, outcomes derived from the JETS e-portfolio is based on self-reported procedural data, which may be at risk of trainee selection bias. However, the outcome plots for both groups appear credible, and the use of JAG certification could be argued as a valid and objective endpoint. This limitation will be addressed with the upcoming integration of the National Endoscopy Database with the JETS e-portfolio, which will enable real-time and unbiased acquisition of lifetime procedure counts during endoscopy training[23]. Finally, the study assumed that trainees who logged procedures onto the JETS e-portfolio after the SPRINT date continued their training during the whole follow up period, and that all intended to reach the milestones of 200 procedures and OGD certification. As such, the analysis was performed on an “intention-to-treat” basis. We excluded trainees who performed no procedures after the date of SPRINT course, to remove those who did not pursue training. However, if there were trainees who ceased training subsequently or had prolonged breaks in training during follow-up, then these will remain included in the analysis, which may underestimate the outcomes measured.

In the face of the upcoming reforms to UK gastroenterology training[5], it is imperative for training programmes to ensure that endoscopy training has been sufficiently optimised. Our study shows that an induction programme for novices in endoscopy is feasible and implementable, can increase trainee confidence, and can shorten the time required to achieve competence for independent practice (JAG certification). Educators should evaluate the effect of educational interventions across training pathways to understand the longer-term outcomes of training. This pilot study provides promising data in support of augmented SBT induction, paving the way for larger and more robust future studies incorporating objective assessments of specific technical and non-technical skills[24], which will better determine its impact on trainee and patient outcomes.

ARTICLE HIGHLIGHTS
Research background

Pre-clinical simulation-based training (SBT) in endoscopy has been shown to augment trainee performance in the short-term, but longer-term data are lacking. The EndoSim (Surgical Science, Gothenburg) is a novel endoscopic virtual reality simulator which incorporates a customisable SBT curriculum and generates task-specific metrics, but has not been validated.

Research motivation

In the United Kingdom, there is no standardised endoscopy SBT induction programme available prior to real-world, patient-based endoscopy training. The Structured PRogramme of INduction and Training (SPRINT) is a two-day gastroscopy induction course combining theory and SBT. We aimed to evaluate: (1) Whether the EndoSim simulator could differentiate between endoscopists of different experience (trainees vs experts); (2) Whether SPRINT improves trainee confidence in technical skills; and (3) Whether SPRINT impacted on longer term trainee outcomes.

Research methods

This prospective study had three components. First, computerised metrics generated by EndoSim were compared between trainees (n = 20) and experts (n = 6) to explore discriminative validity. Second, trainee feedback was acquired immediately pre- and post-course, and pairwise comparisons performed to assess impact of SPRINT on trainee confidence in technical skills. Third, a case-control study was performed to assess the impact of SPRINT on long-term outcomes (16-mo post-course period), which comprised: (1) Rates of unassisted procedural completion; (2) Post-course procedural exposure; (3) Procedural discomfort; (4) Sedation practice; and (5) Rates of gastroscopy certification. Controls matched for gastroscopy experience and study outcomes were derived from the United Kingdom training e-portfolio.

Research results

Of the modules relevant to gastroscopy training, a statistically significant difference was observed in 64% of EndoSIM metrics. Post-SPRINT, trainee confidence increased in all technical skills surveyed. For the case-control element, 15 cases and 24 controls were included, with mean procedure counts of 10 and 3 (P = 0.739) pre-SPRINT. Post-SPRINT, no significant differences between the groups were detected in long-term D2 intubation rates (P = 0.332) or discomfort scores (P = 0.090). However, the cases had a significantly higher rate of unsedated procedures than controls post-SPRINT (58% vs 44%, P = 0.018), which was maintained over the subsequent 200 procedures. Cases tended to perform procedures at a greater frequency than controls in the post-SPRINT period (median: 16.2 vs 13.8 per mo, P = 0.051), resulting in a significantly greater proportion of cases achieving gastroscopy certification by the end of follow up (75% vs 36%, P = 0.017).

Research conclusions

In this pilot study, attendees of the SPRINT cohort tended to perform more procedures and achieved gastroscopy certification earlier than controls, although no significant differences were shown in unassisted D2 intubation rates. These data support the role for wider evaluation of pre-clinical induction involving SBT.

Research perspectives

An induction programme for trainees in endoscopy is feasible and implementable, can increase trainee confidence, and can shorten the time required to achieve competence for independent practice (i.e., certification). This pilot study provides promising data in support of augmented SBT induction, paving the way for phased implementation and larger real-world studies incorporating objective competency assessment tools to compare progress in specific technical and non-technical skills.

Footnotes

Manuscript source: Invited manuscript

Specialty type: Gastroenterology and hepatology

Country of origin: United Kingdom

Peer-review report classification

Grade A (Excellent): 0

Grade B (Very good): B, B

Grade C (Good): C, C

Grade D (Fair): 0

Grade E (Poor): 0

P-Reviewer: Figueiredo EG, Hu B, Rawat K, Tsou YK, S-Editor: Wang JL L-Editor: A E-Editor: Liu JH

References
1.  Siau K, Hawkes ND, Dunckley P. Training in Endoscopy. Curr Treat Options Gastroenterol. 2018;16:345-361.  [PubMed]  [DOI]
2.  Siau K, Green JT, Hawkes ND, Broughton R, Feeney M, Dunckley P, Barton JR, Stebbing J, Thomas-Gibson S. Impact of the Joint Advisory Group on Gastrointestinal Endoscopy (JAG) on endoscopy services in the UK and beyond. Frontline Gastroenterol. 2019;10:93-106.  [PubMed]  [DOI]
3.  Ward ST, Hancox A, Mohammed MA, Ismail T, Griffiths EA, Valori R, Dunckley P. The learning curve to achieve satisfactory completion rates in upper GI endoscopy: an analysis of a national training database. Gut. 2017;66:1022-1033.  [PubMed]  [DOI]
4.  Siau K, Anderson JT, Valori R, Feeney M, Hawkes ND, Johnson G, McKaig BC, Pullan RD, Hodson J, Wells C, Thomas-Gibson S, Haycock AV, Beales ILP, Broughton R, Dunckley P; Joint Advisory Group on Gastrointestinal Endoscopy (JAG). Certification of UK gastrointestinal endoscopists and variations between trainee specialties: results from the JETS e-portfolio. Endosc Int Open. 2019;7:E551-E560.  [PubMed]  [DOI]
5.  Clough J, FitzPatrick M, Harvey P, Morris L. Shape of Training Review: an impact assessment for UK gastroenterology trainees. Frontline Gastroenterol. 2019;10:356-363.  [PubMed]  [DOI]
6.  Qiao W, Bai Y, Lv R, Zhang W, Chen Y, Lei S, Zhi F. The effect of virtual endoscopy simulator training on novices: a systematic review. PLoS One. 2014;9:e89224.  [PubMed]  [DOI]
7.  Singh S, Sedlack RE, Cook DA. Effects of simulation-based training in gastrointestinal endoscopy: a systematic review and meta-analysis. Clin Gastroenterol Hepatol. 2014;12:1611-23.e4.  [PubMed]  [DOI]
8.  Khan R, Plahouras J, Johnston BC, Scaffidi MA, Grover SC, Walsh CM. Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev. 2018;8:CD008237.  [PubMed]  [DOI]
9.  Siau K, Hodson J, Valori RM, Ward ST, Dunckley P. Performance indicators in colonoscopy after certification for independent practice: outcomes and predictors of competence. Gastrointest Endosc. 2019;89:482-492.e2.  [PubMed]  [DOI]
10.  Vassiliou MC, Dunkin BJ, Fried GM, Mellinger JD, Trus T, Kaneva P, Lyons C, Korndorffer JR, Ujiki M, Velanovich V, Kochman ML, Tsuda S, Martinez J, Scott DJ, Korus G, Park A, Marks JM. Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Surg Endosc. 2014;28:704-711.  [PubMed]  [DOI]
11.  Mehta T, Dowler K, McKaig BC, Valori RM, Dunckley P. Development and roll out of the JETS e-portfolio: a web based electronic portfolio for endoscopists. Frontline Gastroenterol. 2011;2:35-42.  [PubMed]  [DOI]
12.  Siau K, Crossley J, Dunckley P, Johnson G, Feeney M, Hawkes ND, Beales ILP; Joint Advisory Group on Gastrointestinal Endoscopy (JAG). Direct observation of procedural skills (DOPS) assessment in diagnostic gastroscopy: nationwide evidence of validity and competency development during training. Surg Endosc. 2020;34:105-114.  [PubMed]  [DOI]
13.  Siau K, Dunckley P, Valori R, Feeney M, Hawkes ND, Anderson JT, Beales ILP, Wells C, Thomas-Gibson S, Johnson G; Joint Advisory Group on Gastrointestinal Endoscopy (JAG). Changes in scoring of Direct Observation of Procedural Skills (DOPS) forms and the impact on competence assessment. Endoscopy. 2018;50:770-778.  [PubMed]  [DOI]
14.  Cohen J, Cohen SA, Vora KC, Xue X, Burdick JS, Bank S, Bini EJ, Bodenheimer H, Cerulli M, Gerdes H, Greenwald D, Gress F, Grosman I, Hawes R, Mullin G, Schnoll-Sussman F, Starpoli A, Stevens P, Tenner S, Villanueva G. Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc. 2006;64:361-368.  [PubMed]  [DOI]
15.  Ferlitsch A, Glauninger P, Gupper A, Schillinger M, Haefner M, Gangl A, Schoefl R. Evaluation of a virtual endoscopy simulator for training in gastrointestinal endoscopy. Endoscopy. 2002;34:698-702.  [PubMed]  [DOI]
16.  Sedlack RE. Validation of computer simulation training for esophagogastroduodenoscopy: Pilot study. J Gastroenterol Hepatol. 2007;22:1214-1219.  [PubMed]  [DOI]
17.  Shirai Y, Yoshida T, Shiraishi R, Okamoto T, Nakamura H, Harada T, Nishikawa J, Sakaida I. Prospective randomized study on the use of a computer-based endoscopic simulator for training in esophagogastroduodenoscopy. J Gastroenterol Hepatol. 2008;23:1046-1050.  [PubMed]  [DOI]
18.  Ende A, Zopf Y, Konturek P, Naegel A, Hahn EG, Matthes K, Maiss J. Strategies for training in diagnostic upper endoscopy: a prospective, randomized trial. Gastrointest Endosc. 2012;75:254-260.  [PubMed]  [DOI]
19.  Ferlitsch A, Schoefl R, Puespoek A, Miehsler W, Schoeniger-Hekele M, Hofer H, Gangl A, Homoncik M. Effect of virtual endoscopy simulator training on performance of upper gastrointestinal endoscopy in patients: a randomized controlled trial. Endoscopy. 2010;42:1049-1056.  [PubMed]  [DOI]
20.  Di Giulio E, Fregonese D, Casetti T, Cestari R, Chilovi F, D'Ambra G, Di Matteo G, Ficano L, Delle Fave G. Training with a computer-based simulator achieves basic manual skills required for upper endoscopy: a randomized controlled trial. Gastrointest Endosc. 2004;60:196-200.  [PubMed]  [DOI]
21.  Grover SC, Scaffidi MA, Khan R, Garg A, Al-Mazroui A, Alomani T, Yu JJ, Plener IS, Al-Awamy M, Yong EL, Cino M, Ravindran NC, Zasowski M, Grantcharov TP, Walsh CM. Progressive learning in endoscopy simulation training improves clinical performance: a blinded randomized trial. Gastrointest Endosc. 2017;86:881-889.  [PubMed]  [DOI]
22.  Biswas S, Alrubaiy L, China L; British Society of Gastroenterology Trainees’ Section, Lockett M, Ellis A, Hawkes N. Trends in UK endoscopy training in the BSG trainees' national survey and strategic planning for the future. Frontline Gastroenterol. 2018;9:200-207.  [PubMed]  [DOI]
23.  Lee TJ, Siau K, Esmaily S, Docherty J, Stebbing J, Brookes MJ, Broughton R, Rogers P, Dunckley P, Rutter MD. Development of a national automated endoscopy database: The United Kingdom National Endoscopy Database (NED). United European Gastroenterol J. 2019;7:798-806.  [PubMed]  [DOI]
24.  Siau K, Kuwai T, Ishaq S. Analysis of learning curves in gastroscopy training: the need for composite measures for defining competence. Gut. 2018;67:1198.  [PubMed]  [DOI]