Impact of a simulation-based induction programme in gastroscopy on trainee outcomes and learning curves

Abstract

BACKGROUND

Pre-clinical simulation-based training (SBT) in endoscopy has been shown to augment trainee performance in the short-term, but longer-term data are lacking.

AIM

To assess the impact of a two-day gastroscopy induction course combining theory and SBT (Structured PRogramme of INduction and Training – SPRINT) on trainee outcomes over a 16-mo period.

METHODS

This prospective case-control study compared outcomes between novice SPRINT attendees and controls matched from a United Kingdom training database. Study outcomes comprised: (1) Unassisted D2 intubation rates; (2) Procedural discomfort scores; (3) Sedation practice; (4) Time to 200 procedures; and (5) Time to certification.

RESULTS

Total 15 cases and 24 controls were included, with mean procedure counts of 10 and 3 (P = 0.739) pre-SPRINT. Post-SPRINT, no significant differences between the groups were detected in long-term D2 intubation rates (P = 0.332) or discomfort scores (P = 0.090). However, the cases had a significantly higher rate of unsedated procedures than controls post-SPRINT (58% vs 44%, P = 0.018), which was maintained over the subsequent 200 procedures. Cases tended to perform procedures at a greater frequency than controls in the post-SPRINT period (median: 16.2 vs 13.8 per mo, P = 0.051), resulting in a significantly greater proportion of cases achieving gastroscopy certification by the end of follow up (75% vs 36%, P = 0.017).

CONCLUSION

In this pilot study, attendees of the SPRINT cohort tended to perform more procedures and achieved gastroscopy certification earlier than controls. These data support the role for wider evaluation of pre-clinical induction involving SBT.

Key Words: Gastroscopy, Esophagogastroduodenoscopy, Endoscopy training, Induction, Competency development, Simulation

Core tip: Simulation-based training has been shown to improve short-term trainee outcomes, but longer-term data on trainee and patient-based outcomes are lacking. A 2-d induction programme covering fundamental theory and hands-on training can improve trainee confidence and shorten the time to achieve gastroscopy certification.

INTRODUCTION

High quality training is a prelude to high quality endoscopy[1]. Within the United Kingdom (UK), quality assurance of endoscopy training is overseen by the Joint Advisory Group on Gastrointestinal Endoscopy (JAG)[2]. For most gastroenterologists, training in endoscopy begins with gastroscopy. The process of gastroscopy training requires considerable time and effort; on average, 187 procedures are required to achieve consistent gastroscopy completion rates (95%+ intubation to the second part of the duodenum – D2)[3] and 282 procedures (1.9 years) to attain JAG certification[4], which is a national requirement for independent practice. With the imminent “Shape of Training” reforms to UK gastroenterology training[5], which proposes to shorten the length of specialist training, endoscopy trainers need to re-evaluate training methods and tools, to deliver evidence-based training pathways which accelerate the development of competency in endoscopic procedures.

Simulation-based training (SBT) provides one solution to this challenge. Modern-day computerised virtual reality (VR) simulators are capable of delivering immersive training without risk of patient harm. The plethora of high-quality evidence attests to the short-term benefits of SBT in augmenting the acquisition of fundamental technical skills such as scope handling and tip control[6-8], which could shorten the learning curve. Additionally, data from colonoscopy training confirm that trainee endoscopists incur more procedural discomfort during earlier stages of training[9]. Despite this, pre-clinical SBT is not readily available within the UK JAG training pathway, where hands-on training typically begins with patient-based gastroscopy at the discretion of supervising trainers. Hence, there is a need for a standardised induction programme which can ensure that beginners are sufficiently armed with the basic skills and knowledge before approaching patient-based training, in line with other international training pathways[10]. Data on the longer-term benefits of SBT on trainee and patient outcomes are lacking.

The Structured PRogramme of INduction and Training (SPRINT) is an induction programme consisting of a structured sequence of theory and hands-on training elements designed to optimise and accelerate the early phase of training in gastroscopy (Table 1). The didactic theory-based seminars are intended to complement SBT and cover fundamental aspects such as endoscope design/handling and basic lesion recognition. The EndoSim (Surgical Science, Gothenburg, Sweden) is a novel endoscopic VR simulator (Figure 1) which incorporates a customisable SBT curriculum and generates task-specific metrics, but has not been validated. In September 2017, a two-day gastroscopy Induction programme combining these foundation knowledge sessions with SBT was provided to medical and surgical trainees from three training deaneries. The primary aim of this training intervention was to assess whether this type of enhanced induction can accelerate competency development in novice trainees. The secondary aim was to evaluate whether EndoSim metrics could distinguish between trainees and experts in order to assess discriminative validity of the simulator.

Table 1 The Structured PRogramme of INduction and Training gastroscopy induction programme.

Time	Programme
8.3	Coffee and registration
9	Welcome and introduction to aims and objectives
9.3	Simulator session 1	Basic handling and scope design
10.2	Basic handling and scope design	Simulator session 1
11.1	Coffee
11.3	Simulator session 2	JAG Certification, appraisal and training lists
12.2	JAG Certification, appraisal and training lists	Simulator session 2
13.1	Lunch
13.4	Simulator session 3	Enhancing the endoscopic image
14.1	Enhancing the endoscopic image	Simulator session 3
15	Coffee
15.2	Simulator session 4	Lesion recognition and assessment skills 1
16.1	Lesion recognition and assessment skills 1	Simulator session 4
17	Round up
8.3	Coffee and registration
9	Welcome and introduction to day 2
9.1	Simulator session 5	Getting the best out of the JETS e-portfolio
10	Getting the best out of the JETS e-portfolio	Simulator session 5
10.5	Coffee
11.1	Simulator session 6	Lesion recognition and assessment skills 2
12	Lesion recognition and assessment skills 2	Simulator session 2
12.5	Lunch
13.2	Simulator session 7	Decision-making and report writing
13.5	Decision-making and report writing	Simulator session 7
14.4	Coffee
15	Simulator session 8	DOPS assessment and improving your skills
15.5	DOPS assessment and improving your skills	Simulator session 8
16.4	Summary and review of course objectives

DOPS: Direct observation of procedural skills.

Figure 1 The surgical science EndoSim simulation module. A: Endoscopy stack; B: Virtual reality views of the duodenum; C: Gastric cardia; D: Endoscope configuration.

MATERIALS AND METHODS

Study design

In this prospective multicentre study, trainees commencing gastroscopy training (ST3) from three UK training deaneries (regions) were enrolled to the augmented SBT induction programme. All trainees completed a structured curriculum of hands-on simulator training lasting a minimum of 3 h with feedback from JAG certified faculty trainers. This study had three components. First, trainee performance on the EndoSim VR simulator was compared to that of the expert faculty (all > 1000 procedures) to explore discriminative validity of the metrics used. To minimise bias, each participant’s first valid attempt, without prior faculty training or feedback, was included. Training was only offered once all trainees had completed the assessment round. The second component was an assessment of the change in self-reported confidence scores following the course, based on questionnaires completed immediately pre- and post-course. The third component of the study assessed the impact of the course on long-term trainee outcomes using a case-control method. In the UK, all trainees are required to log training procedures onto the JAG Endoscopy Training System (JETS) e-portfolio[11]. Participation is mandated to enable certification for independent practice. For this analysis, a cohort of control trainees was selected from non-attenders who had submitted formative assessment data on JETS in the post-course period between September and December 2017. Only trainees with < 50 procedures, and whose first JETS-recorded procedure was less than a year prior to the date of SPRINT were included in the analyses of long-term outcomes, to ensure the levels of experience were similar in cases and controls. In addition, trainees with no gastroscopy procedures logged on JETS post-SPRINT were excluded, since these had not begun hands-on training in the post-course period. Trainee and patient outcomes for each post-course training procedure were extracted from the JETS e-portfolio, with prospective follow-up of outcomes post-SPRINT performed from September 2017 until February 2019 (maximum period of 16 mo).

Study approval

Study approval was granted by JAG Quality Assurance of Training working group. All participants provided written, informed consent for inclusion within the study. Neither the researchers nor Surgical Science had access to the study outcome data over the course of the study. There was no financial incentive to conduct this study.

Study outcomes

Discriminative validity of EndoSim: EndoSim scenarios and computer-generated metrics relevant to the assessment of technical skills in gastroscopy were selected as study outcomes for the first component of the study. Pre-set modules relevant to gastroscopy were selected, which generated skillset-dependent EndoSim metrics. Comparisons were then made between trainees and experts, followed by subgroup analyses within the trainees, comparing novices (< 25 procedures) to those with intermediate experience (25 + procedures).

Impact on self-assessed competence scores: Questionnaires were administered to all trainees both pre- and post-course, which measured self-assessed competency scores in 12 upper GI handling skills domains. These domains were mapped to the formative JAG formative assessment forms which are integrated into UK endoscopy training[12]. Domain scores were given as a rating from 0-10 on a Likert scale (0 = not at all competent, 10 = very competent).

Long-term trainee outcomes: For the analysis of long-term trainee outcomes, the trainee outcomes included the unassisted rate of D2 intubation, i.e., procedures without physical assistance, the volume of training procedures performed post-course, and the time taken to achieve JAG gastroscopy certification. JAG certification involves the composite outcome of attaining a minimum lifetime patient-based gastroscopy count (200+) and satisfactory completion of formative and summative direct observation of procedural skills assessments to objectively demonstrate competence[4,12,13]. Patient outcomes were also explored; these included comparisons in rates of moderate-to-severe discomfort and rates of unsedated procedures.

Statistical analyses

For the first two components of the study, variables were reported as medians and interquartile ranges (IQRs), and compared between groups using Mann-Whitney tests, with Wilcoxon’s tests used for paired comparisons. For the third component, trends in study outcomes (i.e., D2 intubation, sedation and discomfort rates) were evaluated over the 200 post-SPRINT procedures using generalised estimating equation (GEE) models, to account for the non-independence of repeated procedures by the same trainee. Prior to the analysis, the relationship between procedural experience and outcome measures were assessed graphically, and transformations applied, as applicable, to ensure goodness-of-fit. Binary logistic GEE models were then produced, using an autoregressive correlation structure. The trainee group (case/control) and the procedure number post-SPRINT were set as covariates, with an interaction term also included in the model. As such, the analysis allowed for the two groups to have different outcome rates at baseline, but also allowed for the rate of improvement over time to vary between the groups, with the interaction term allowing for comparison of the latter.

The times taken to reach the 200^th procedure and for gastroscopy certification were assessed with Kaplan-Meier plots, with comparisons between cases and controls made using univariable Cox regression models. Trainees who did not achieve these outcomes were censored at the date of their final procedure. The procedure counts in the post-SPRINT period were then compared. To account for the potential loss of follow-up after gastroscopy certification, procedures performed after certification were excluded. Since this resulted in differing durations of follow up across the trainees, the total procedure counts were then divided by the time between the SPRINT course and gastroscopy certification, or to the final procedure date in those without certification, and analysed as an average number of procedures per month. All analyses were performed using SPSS v24 (IBM Corp. Armonk, NY), with P < 0.05 indicating statistical significance.

RESULTS

Participants

In total, 20 trainees and 6 faculty members (experts) attended the SPRINT induction programme. Of these, 10 trainees were classified as novices (< 25 procedures) and 10 as having intermediate experience (25+ procedures). Data from all participants were included in the analyses of EndoSim metrics and self-assessment scores. For the learning curve analyses, trainees were excluded on the basis of: Having no procedures recorded in the JETS e-portfolio (n = 3), > 50 pre-course procedures (n = 1), and first recorded procedure > 1 year before the course (n = 1), leaving 15 cases for analysis. Data for 24 control patients were identified. The majority of cases (93%) and controls (88%) were gastroenterology trainees, with upper gastrointestinal surgical trainees comprising the remainder. There were no significant differences in trainee specialty between groups (P = 0.967). Prior to the date of SPRINT, the average number of procedures performed was similar between the two groups (P = 0.739), with a mean of 10 per trainee in cases and 3 per trainee in controls; 63% of controls and 60% of cases had performed zero procedures prior to the course date.

Discriminative validity of EndoSim metrics

Comparisons of EndoSim metrics between trainees (novice and intermediate) and experts are presented in Table 2. Five gastroscopy-relevant modules were selected comprising: wheel handling, navigation, button handling, photodocumentation and biopsy, each with a variable number of substations. All trainees and faculty successfully completed and passed each station. Trainees could be differentiated from experts for at least one metric on all modules except for the “button handling” station. For the remaining stations, experts could be delineated from trainee performance in terms of efficiency metrics (i.e., total time to complete a task), efficiency of movement (e.g., more conservative wheel and scope rotation and less endoscope tip path length in the Navigation module, fewer collisions against the mucosa) and precision (fewer missed targets and more biopsies within target). Of the 22 metrics relevant to the five modules, 5 (23%) were significantly different between novice and intermediate trainees and 14 (64%) between trainees and experts.

Table 2 Comparisons of module-dependent EndoSim metrics between trainees (stratified into novice and intermediate experience groups) and faculty members.

Module	Metric	Median (IQR)		P value (Expert vs trainee)	Median (IQR)		P value (Novice vs intermediate)
		Expert (n =6)	Trainee (n = 20)		Novice trainee (n = 10)	Intermediate trainee (n = 10)
Module 1: Wheel Handling (4 stations)	Missed targets	3 (1-4)	6 (3-8)	< 0.001	7 (4-9)	6 (2-8)	0.057
	Wheel rotation left/right (Degrees)	257 (42-382)	143 (5-591)	0.463	82 (1-643)	166 (9-575)	0.753
	Wheel rotation up/down (Degrees)	783 (691-916)	764 (606-1173)	0.903	680 (442-1005)	1023 (687-1303)	0.003
	Endoscope rotation (Degrees)	1398 (749-2355)	964 (353-1577)	0.025	886 (350-1404)	1044 (349-1955)	0.350
Module 2: Navigation (3 stations)	Total time (s)	74 (52-104)	104 (79-166)	0.002	161 (108-218)	82 (67-105)	< 0.001
	Wheel rotation left/right (Degrees)	109 (40-305)	138 (3-757)	0.826	391 (3-1648)	99 (2-549)	0.143
	Wheel rotation up/down (Degrees)	888 (680-1108)	1232 (934-1868)	0.001	1268 (958-1737)	1224 (931-2046)	0.641
	Endoscope rotation (Degrees)	1120 (933-1865)	1770 (1313-2334)	0.007	1847 (1258-2571)	1722 (1357-2258)	0.503
	Endoscope tip path length (cm)	228 (179-306)	324 (251-411)	0.002	357 (280-489)	280 (239-356)	0.028
Module 3: Button Handling (3 stations)	Missed targets (number)	2 (0-4)	2 (1-4)	0.623	2 (1-5)	1.5 (1-4)	0.805
	Unnecessary button presses (number)	2 (0-4)	2 (1-4)	0.270	2 (1-5)	2 (1-4)	0.963
	Missed dirt (number)	1 (1-2)	1 (1-2)	0.944	1 (0-2)	1 (1-2)	0.429
Module 4: Photo (4 stations)	Total time (s)	151 (121-192)	313 (230-377)	< 0.001	328 (235-404)	269 (179-361)	0.054
	Stomach visualized (%)	93% (79%-99%)	100 (96%-100%)	< 0.001	99% (94%-100%)	100% (97%-100%)	0.070
	Duodenum visualized (%)	63% (52%-74%)	63% (53-72%)	0.855	62% (51%-68%)	66% (58%-74%)	0.088
	Collisions against mucosa (number)	8 (5-12)	13 (9-16)	< 0.001	13 (11-20)	12 (8-15)	0.090
	Targets photographed (%)	100% (100%-100%)	100% (100%-100%)	0.495	100% (100%-100%)	100% (100%-100%)	0.302
Module 5: Biopsy (3 stations)	Total time (s)	182 (163-217)	340 (249-463)	< 0.001	446 (331-522)	299 (215-389)	0.001
	Targets biopsied	100% (100%-100%)	100% (50-100%)	0.010	100% (38%-100%)	100% (50%-100%)	0.546
	Biopsies outside any target (number)	0 (0-2)	4 (2-9)	< 0.001	3 (2-8)	4 (2-11)	0.548
	Collisions against mucosa (number)	7 (4-11)	9 (7-13)	0.030	12 (9-23)	7 (6-11)	< 0.001
	Movement with tool (cm)	25 (17-53)	72 (36-183)	0.002	73 (29-183)	70 (42-179)	0.910

Data are presented as medians (interquartile ranges), with P values derived from Mann-Whitney tests. Bold P values are significant at P < 0.05. IQR: Interquartile ranges.

Self-assessment scores

The 20 trainees attending SPRINT reported their confidence in 12 different skills both pre- and post-course. Across these skills, the median confidence score ranged from 3-5 on the pre-course questionnaire (Table 3). After the course, confidence in all 12 skills increased significantly (all P < 0.001), with medians ranging from 7-9.

Table 3 Self-reported scores pre- and post-course.

Skill	Median confidence score (IQR)		P value
	Pre-course	Post-course
Tip control	5 (2-7)	8 (7-9)	< 0.001
Torque steering	5 (2-6)	8 (7-9)	< 0.001
Intubation	3 (0-7)	7 (6-9)	< 0.001
Oesophagus to pylorus	5 (1-8)	9 (7-9)	< 0.001
Pyloric intubation	4 (0-7)	8 (7-9)	< 0.001
D2 intubation	3 (0-6)	7 (5-9)	< 0.001
Duodenal withdrawal	4 (0-7)	8 (5-9)	< 0.001
J manoeuvre	5 (1-8)	8 (7-9)	< 0.001
Retroflexed views	5 (2-7)	8 (6-9)	< 0.001
Overall visualisation	5 (2-7)	8 (7-9)	< 0.001
Image taking	4 (1-6)	8 (7-8)	< 0.001
Use of accessories	3 (0-5)	8 (6-9)	< 0.001

Analysis is based on the n = 20 who attended the course. P values are from Wilcoxon’s tests, with bold P values significant at P < 0.05. IQR: Interquartile range.

Trends in procedural outcomes by course attendance

Preliminary analysis of the trends in unassisted D2 intubation rates found these to increase rapidly over the initial post-course procedures, with the rate of improvement slowing after 25-50 procedures and flattening subsequently as the D2 intubation rates neared 90%. Due to this rapid change in gradient, a binary logistic regression model with the lifetime procedure number as a covariate resulted in a poor fit. As such, lifetime procedure counts were log₂-transformed, after adding 10, which improved the goodness-of-fit of the model.

This model was then applied to the cases and controls separately, using a GEE approach, to compare the rate of improvement between groups (Table 4, Figure 2A). This showed comparable baseline unassisted D2 intubation rates, with estimates of 28% for cases and 35% for controls at the first procedure post-SPRINT (P = 0.332). Unassisted D2 intubation rates improved with experience, with a doubling of the procedure count associated with an odds ratio of 1.99 in cases (P < 0.001) and 1.74 (P < 0.001) in controls (Table 4). However, no significant difference was detected between these gradients (P = 0.205). Hence, the rates of improvement in unassisted D2 intubation rates by procedure number were comparable between cases and controls.

Figure 2 Plots of outcome rates by post-course procedure number, stratified according to cases and controls. A: Unassisted D2 intubation rates; B: Rates of moderate-severe discomfort; C: Unsedated procedures. Trendlines are extrapolated from generalised estimating equation models, as described in Table 4.

Table 4 Generalised estimating equation models of procedure outcomes in cases and controls.

	Cases		Controls		P value (Case vs Control)
	Odds ratio (95%CI)	P value	Odds ratio (95%CI)	P value
Unassisted D2 intubation rates
Intercept	0.51 (0.13 – 1.98)	-	-	-	0.332
Gradient (per Doubling of OGD count)	1.99 (1.69 – 2.34)	< 0.001	1.74 (1.53 – 1.98)	< 0.001	0.205
Moderate-severe discomfort
Intercept	0.42 (0.15 - 1.15)	-	-	-	0.09
Gradient (per 10 procedures)	0.97 (0.88 - 1.07)	0.526	0.92 (0.85 - 1.00)	0.044	0.421
Unsedated procedures
Intercept	1.63 (1.09 – 2.46)	-	-	-	0.018
Gradient (per 10 procedures)	0.99 (0.97 – 1.01)	0.28	1.00 (0.98 – 1.02)	0.973	0.445

Results are from generalised estimating equation models of the 200 procedures after the date of the course for each trainee. Each model contained the trainee group (case-control) and the procedure number as covariates, along with an interaction term. As such, the intercept represents the baseline difference between the case and control groups. The gradient represents the change in the outcome rate with increasing experience, with separate gradients reported for the case and control groups. For unassisted D2 rates, the procedure number was log-transformed in the model, hence the resulting coefficients were anti-logged, and gradients were reported per two-fold increase in procedure count. For the discomfort and unsedated procedures outcomes, gradients are reported per 10 additional procedures. The final column compares the gradients between groups, using the P value from the interaction term in the model. Bold P values are significant at P < 0.05.

Of the other outcomes considered, no significant differences in the rates of moderate-severe discomfort were detected between the case and control groups (Figure 2B). However, significant differences in the proportions of unsedated procedures were observed (Figure 2C). At baseline, the cases performed a significantly greater proportion of procedures without sedation (odds ratio: 1.63, 95%CI: 1.09–2.46, P = 0.018), with 58% of the first ten procedures by cases being unsedated, compared to 44% of those by controls. This difference between the groups was then sustained over the subsequent procedures (interaction term: P = 0.445).

Times to performance milestones

Over the post-SPRINT follow-up period of 16 mo, 13/15 (87%) cases and 15/24 (63%) controls reached a lifetime procedure count of 200. The Kaplan-Meier estimated median time to the 200^th procedure (Figure 3A) did not differ significantly (P = 0.190) between cases (10.6 mo) and controls (12.1 mo). Gastroscopy certification was achieved in 11/15 (73%) cases and 7/24 controls (29%). Delegates achieved certification after a median time of 14 mo post-SPRINT, which was significantly earlier than controls (Figure 3B, P = 0.017), for whom the rate did not reach 50% (i.e., the median time was > 16 mo). By the end of follow up (i.e., 16 mo), the Kaplan-Meier estimated certification rates were 75% vs 36% in cases vs controls.

Figure 3 Kaplan-Meier curves of time to 200 procedures (A) and time to gastroscopy certification (B).

Post-course procedure counts

Cases performed a median of 16.2 (IQR: 13.8-20.8) procedures per month post-SPRINT, which was higher than the 13.8 (IQR: 9.2-16.7) observed in controls, although this missed statistical significance (P = 0.051).

DISCUSSION

In this small prospective case-control pilot study, trainees who attended a two-day hands-on gastroscopy induction course involving basic theory and SBT showed no significant difference in the learning curve to achieve unassisted D2 intubation. However, attenders achieved JAG certification earlier than peers from the control group, which may be explained by the tendency to perform more post-course procedures (P = 0.051). These results provide real-world data on the durability of an SBT induction programme on trainee and patient outcomes.

To account of the possibility of trainees attempting harder to complete an examination, discomfort and sedation-based outcomes were also explored. Course attenders performed more post-course procedures without sedation (P = 0.018), but without a significant difference in rates of moderate-severe discomfort (P = 0.090). To our knowledge, the only other publication which assessed the durability of SBT over 200 procedures was the randomised controlled trial (RCT) by Cohen et al[14] in the context of colonoscopy training. Trainees allocated to pre-clinical SBT demonstrated superior technical and cognitive outcomes during early stages of training, but this effect dissipated after 100 procedures. Previous RCTs on gastroscopy training have assessed post-SBT outcomes after 3-4 wk[15,16], or after 2-60 procedures of patient-based training[17-20], with the majority showing improvements in favour of SBT. Study protocols have also varied with regard to the duration of SBT exposure and training structure. Di Giulio et al[20] found that trainees randomised to 10 h of SBT performed better at unassisted D2 intubation, retroflexion and landmark identification over 20 patient-based procedures (88% vs 70%, P < 0.001). In another RCT, trainees who underwent 2 h of SBT demonstrated improved D2 intubation over 10 procedures, but no difference in patient discomfort scores[19].

Not all studies have associated SBT with improved outcomes. The Sedlack[16] study found no benefit in trainee gastroscopy outcomes after 6 h of SBT. Concerns over the face validity of the simulator was cited as a possible explanation. Our study presents novel evidence of discriminant validity of the EndoSim platform and response process validity in form of trainee feedback and the improvements in self-confidence scores. Although we found no significant difference between cases and controls in unassisted D2 intubation rates by procedure count, the rates at which trainees acquired competence, as evidenced by time-to-competency endpoints, was found in favour of SPRINT course attenders. It is recognised that coaching and feedback[8], coupled with a structured SBT curriculum[21], and a minimum exposure period to SBT[1], may be required to unlock the full potential of SBT. Notably, the difference between groups in rates of unsedated procedures may be another confounding factor.

Our study provides novel data from the perspective of UK-based endoscopy training which has its imperfections[4,22]. In addition to the lack of standardised SBT-curricula, training occurs within endoscopy units which face the perennial dilemma of balancing service capacity with list reductions to accommodate novice trainees. Trainees often face competing commitments, e.g., from on-call rotas, ward and clinic duties, and may have to compete for training with other specialties[22]. It is possible that the improvements in trainee confidence derived from SPRINT may empower trainees to train on unsedated procedures and adopt a more proactive training stance, thereby leading to greater acquisition of training experience and shorter times to certification.

Our study had several noteworthy limitations. First, this was a non-randomised pilot study. Despite the similar numbers of pre-course procedures in the two groups, we cannot exclude differences in training provisions within the training regions for cases and controls. Second, this pilot study, with its 15 eligible cases, may be underpowered to detect statistically significant differences in long-term study outcomes. No formal power calculation was performed prior to study commencement, as the included trainees consisted of a convenience sample of SPRINT course attendees, hence there was no scope to increase the sample size. Third, not all trainees were fully novices, with approximately 40% having some degree of gastroscopy experience. However, this was generally limited to a small number of procedures and did not differ significantly between cases and controls. Fourth, the validity of EndoSim was not rigorously appraised, as the primary intention was to provide training and to assess longer term outcomes. Further evaluation is required to appraise face validity, i.e., realism, and comparisons of EndoSim performance over time. Fifth, owing to the comparisons of trainee and faculty performance of the EndoSim simulator, modules were initially performed by trainees without coaching or feedback, which are pivotal for skills acquisition with SBT. Faculty experts were unfamiliar to EndoSim and did not receive pre-course training. These factors may have compromised the effectiveness of hands-on technical skills training. Sixth, outcomes derived from the JETS e-portfolio is based on self-reported procedural data, which may be at risk of trainee selection bias. However, the outcome plots for both groups appear credible, and the use of JAG certification could be argued as a valid and objective endpoint. This limitation will be addressed with the upcoming integration of the National Endoscopy Database with the JETS e-portfolio, which will enable real-time and unbiased acquisition of lifetime procedure counts during endoscopy training[23]. Finally, the study assumed that trainees who logged procedures onto the JETS e-portfolio after the SPRINT date continued their training during the whole follow up period, and that all intended to reach the milestones of 200 procedures and OGD certification. As such, the analysis was performed on an “intention-to-treat” basis. We excluded trainees who performed no procedures after the date of SPRINT course, to remove those who did not pursue training. However, if there were trainees who ceased training subsequently or had prolonged breaks in training during follow-up, then these will remain included in the analysis, which may underestimate the outcomes measured.

In the face of the upcoming reforms to UK gastroenterology training[5], it is imperative for training programmes to ensure that endoscopy training has been sufficiently optimised. Our study shows that an induction programme for novices in endoscopy is feasible and implementable, can increase trainee confidence, and can shorten the time required to achieve competence for independent practice (JAG certification). Educators should evaluate the effect of educational interventions across training pathways to understand the longer-term outcomes of training. This pilot study provides promising data in support of augmented SBT induction, paving the way for larger and more robust future studies incorporating objective assessments of specific technical and non-technical skills[24], which will better determine its impact on trainee and patient outcomes.

ARTICLE HIGHLIGHTS

Research background

Pre-clinical simulation-based training (SBT) in endoscopy has been shown to augment trainee performance in the short-term, but longer-term data are lacking. The EndoSim (Surgical Science, Gothenburg) is a novel endoscopic virtual reality simulator which incorporates a customisable SBT curriculum and generates task-specific metrics, but has not been validated.

Research motivation

In the United Kingdom, there is no standardised endoscopy SBT induction programme available prior to real-world, patient-based endoscopy training. The Structured PRogramme of INduction and Training (SPRINT) is a two-day gastroscopy induction course combining theory and SBT. We aimed to evaluate: (1) Whether the EndoSim simulator could differentiate between endoscopists of different experience (trainees vs experts); (2) Whether SPRINT improves trainee confidence in technical skills; and (3) Whether SPRINT impacted on longer term trainee outcomes.

Research methods

This prospective study had three components. First, computerised metrics generated by EndoSim were compared between trainees (n = 20) and experts (n = 6) to explore discriminative validity. Second, trainee feedback was acquired immediately pre- and post-course, and pairwise comparisons performed to assess impact of SPRINT on trainee confidence in technical skills. Third, a case-control study was performed to assess the impact of SPRINT on long-term outcomes (16-mo post-course period), which comprised: (1) Rates of unassisted procedural completion; (2) Post-course procedural exposure; (3) Procedural discomfort; (4) Sedation practice; and (5) Rates of gastroscopy certification. Controls matched for gastroscopy experience and study outcomes were derived from the United Kingdom training e-portfolio.

Research results

Of the modules relevant to gastroscopy training, a statistically significant difference was observed in 64% of EndoSIM metrics. Post-SPRINT, trainee confidence increased in all technical skills surveyed. For the case-control element, 15 cases and 24 controls were included, with mean procedure counts of 10 and 3 (P = 0.739) pre-SPRINT. Post-SPRINT, no significant differences between the groups were detected in long-term D2 intubation rates (P = 0.332) or discomfort scores (P = 0.090). However, the cases had a significantly higher rate of unsedated procedures than controls post-SPRINT (58% vs 44%, P = 0.018), which was maintained over the subsequent 200 procedures. Cases tended to perform procedures at a greater frequency than controls in the post-SPRINT period (median: 16.2 vs 13.8 per mo, P = 0.051), resulting in a significantly greater proportion of cases achieving gastroscopy certification by the end of follow up (75% vs 36%, P = 0.017).

Research conclusions

In this pilot study, attendees of the SPRINT cohort tended to perform more procedures and achieved gastroscopy certification earlier than controls, although no significant differences were shown in unassisted D2 intubation rates. These data support the role for wider evaluation of pre-clinical induction involving SBT.

Research perspectives

An induction programme for trainees in endoscopy is feasible and implementable, can increase trainee confidence, and can shorten the time required to achieve competence for independent practice (i.e., certification). This pilot study provides promising data in support of augmented SBT induction, paving the way for phased implementation and larger real-world studies incorporating objective competency assessment tools to compare progress in specific technical and non-technical skills.