Minireviews Open Access
Copyright ©The Author(s) 2018. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastroenterol. Dec 28, 2018; 24(48): 5439-5445
Published online Dec 28, 2018. doi: 10.3748/wjg.v24.i48.5439
Virtual reality simulation in endoscopy training: Current evidence and future directions
Tahrin Mahmood, Michael Anthony Scaffidi, Rishad Khan, Samir Chandra Grover
Tahrin Mahmood, Michael Anthony Scaffidi, Rishad Khan, Samir Chandra Grover, Division of Gastroenterology, St. Michael’s Hospital, University of Toronto, Toronto M5B 1W8, Canada
ORCID number: Tahrin Mahmood (0000-0001-9013-8166); Michael Anthony Scaffidi (0000-0003-2068-6655); Rishad Khan (0000-0002-5090-7685); Samir Chandra Grover (0000-0003-3392-1220).
Author contributions: Mahmood T, Scaffidi MA, Khan R, and Grover SC drafted the manuscript, critically appraised it for important intellectual content, and approved the final version of this manuscript.
Conflict-of-interest statement: No authors have any conflicts of interest to report of relevance to the subject matter of this manuscript.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Corresponding author: Samir Chandra Grover, MD, MEd, FRCPC, Assistant Professor, Division of Gastroenterology, St. Michael’s Hospital, University of Toronto, 30 Bond Street, Toronto M5B 1W8, Canada. samir.grover@utoronto.ca
Telephone: +1-416-864-5628 Fax: +1-416-864-5882
Received: September 29, 2018
Peer-review started: September 29, 2018
First decision: October 26, 2018
Revised: November 22, 2018
Accepted: November 30, 2018
Article in press: December 1, 2018
Published online: December 28, 2018

Abstract

Virtual reality simulation is becoming the standard when beginning endoscopic training. It offers various benefits including learning in a low-stakes environment, improvement of patient safety and optimization of valuable endoscopy time. This is a review of the evidence surrounding virtual reality simulation and its efficacy in teaching endoscopic techniques. There have been 21 randomized controlled trials (RCTs) that have investigated virtual reality simulation as a teaching tool in endoscopy. 10 RCTs studied virtual reality in colonoscopy, 3 in flexible sigmoidoscopy, 5 in esophagogastroduodenoscopy, and 3 in endoscopic retrograde cholangiopancreatography. RCTs reported many outcomes including distance advanced in colonoscopy, comprehensive assessment of technical and non-technical skills, and patient comfort. Generally, these RCTs reveal that trainees with virtual reality simulation based learning improve in all of these areas in the beginning of the learning process. Virtual reality simulation was not effective as a replacement of conventional teaching methods. Additionally, feedback was shown to be an essential part of the learning process. Overall, virtual reality endoscopic simulation is emerging as a necessary augment to conventional learning given the ever increasing importance of patient safety and increasingly valuable endoscopy time; although work is still needed to study the nuances surrounding its integration into curriculum.

Key Words: Endoscopy, Gastrointestinal/education, Endoscopy, Gastrointestinal/standards, Simulation, Educational measurement, Clinical competence/standards, Competency-based medical education

Core tip: There is substantial evidence to support that virtual reality simulation is an excellent augment to the traditional apprenticeship model in learning endoscopic procedures. Further work is still needed to study the nuances surrounding its integration into curriculum.



INTRODUCTION

Educators in gastroenterology are increasingly integrating simulation-based training (SBT) into endoscopy curricula. SBT allows for a learner-centered environment in which novices can engage in deliberate practice without the fear of making mistakes or harming a patient[1-4]. The application of SBT to endoscopy has manifested as a variety of model systems, including mechanical models, animal models, live ex-vivo models, and virtual reality (VR) computer simulators. Among these, VR simulators have become the most commonly tested model system[5,6]. In this narrative review, we explore the current state of evidence for the use of VR SBT in endoscopic training.

ENDOSCOPY TRAINING
Apprenticeship model

The apprenticeship model of learning is based on an expert endoscopist (coach) teaching a trainee (apprentice) endoscopic skills. The apprenticeship helps the trainee progress from observation to participation, and finally to independence with progressive responsibilities[7,8]. The apprenticeship model is based in situated learning theory, which holds that a skill must be learned in the authentic context where it will be applied[9]. During training, the trainee’s responsibility and independence increases as he/she accumulates experience and skill (Figure 1)[3,5]. Experienced endoscopists intervene if they deem that the trainee will be unable to complete the procedure safely, and trainees are expected to demonstrate certain milestones before progressing to the next steps.

Figure 1
Figure 1 Progression of learning in the apprenticeship model. The apprenticeship model relies on the trainee modelling after an expert, followed by experimentation and approximation that eventually lead to increased autonomy, adaptation to new problems and eventual mastery.

There are, however, several important disadvantages of the apprenticeship model. First, there may be an increased risk of adverse events for patients. For example, one study found that the number of complications of endoscopy are significantly increased in July or August, when the training programs start with their new trainees[10]. Similarly, a study by Matharoo et al[11] showed that trainee related factors resulted in as many patient safety incidents as sedation with no oxygen saturation monitor. Second, the staff endoscopist has to completely give up the control of the endoscope in order for the trainee to learn[5,12]. As the trainee does not have the experience to identify and appropriately manage the findings, the pathological findings, which are intermittent and unpredictable, maybe mismanaged[5,12]. Third, training endoscopies take longer to complete, adding to the already strained availability of the endoscopy suite[5]. Fourth, the feedback given to each trainee depends on the staff endoscopist, and therefore may vary[13,14]. Finally, there is a lack of continuity as the trainees are expected to work with different staff endoscopists, who may not understand the trainee’s level of competence as a whole[8].

Simulation based training

Given these drawbacks, the endoscopy educators increasingly use SBT prior to first patient contact. SBT provides the trainee with a simulated model of reality to help the trainee understand the skills required for clinical practice[6,15]. The simulated model can be made physically (i.e., bench top) or with the help of technology using VR[6,13,15]. SBT is based on the constructivist learning model, which holds that learning is constructed by the learner, rather than transferred to the learner; this renders the context of the situation not as important[16]. In endoscopic training, SBT has gained attention because it provides a way of training without the risk of adverse events to patients, and allows for standardization of feedback through the simulation[5,17,18]. Virtual reality simulators can model endoscopy by using an endoscope that is inserted into a computer-based module which displays the gastrointestinal lumen on a screen, and provides visual and tactile feedback related to the procedure[13].

CURRENT STATE OF EVIDENCE IN VIRTUAL-REALITY SBT IN ENDOSCOPY

Previous systematic reviews and meta-analyses have effectively analyzed the results of previous studies, including a report of internal validity of the existing literature[5,17,18]. Briefly, to date there have been 21 major RCTs investigating VR simulation and its role in training (Table 1). 10 of these have studies investigated VR in colonoscopy, 5 in organic gastrointestinal disease (OGD), 3 in flexible sigmoidoscopy, and 3 in endoscopic retrograde cholangiopancreatography ERCP (Table 1)[19-39].

Table 1 Summary of randomized controlled trials.
Ref.Sample sizeInterventionOutcomes
Flexible Sigmoidoscopy
Tuggy et al[19]10Group 1: VR simulationGroup 1 performed better: Faster mean completion time (323 s vs 654 s), lower directional errors (1.6% vs 8.6%), higher % of colon visualized (79% vs 45%)
Group 2: No simulation
Gerson et al[20]16Group 1: VR simulationGroup 1 performed worse: Lower mean score (2.9 vs 3.8 out of 5), lower cases completed independently (29% vs 72%), lower retroflexion completed (56% vs 84%); average time, patient satisfaction did not differ
Group 2: Conventional teaching
Sedlack et al[21]38Group 1: VR simulationGroup 1 performed better: Higher patient comfort; procedural skills (independence, identifying pathology, landmarks, performing biopsies, adequate visualization) did not differ
Group 2: Conventional teaching
Colonoscopy
Sedlack et al[22]8Group 1: VR simulationGroup 1 performed better in first 30 procedures: High depth of unassisted insertion, higher % of procedures completed independently (64.1% vs 56.3%), high scores on other measures such as ability to insert in a safe manner, adequality visualize mucosa, identify landmarks; mean time to reach maximum insertion did not differ
Group 2: No simulation
Ahlberg et al[23]12Group 1: VR simulationGroup 1 performed better: Higher rates of insertion to cecum (52% vs 19%), shorter procedure time (30 min vs 40 min), less patient discomfort
Group 2: No simulation
Cohen et al[24]45Group 1: VR simulationGroup 1 performed better: Higher competence scores as judged by ability to reach the transverse colon and cecum without assistance (92.7% vs 90.9% by Session 10); patient comfort did not differ
Group 2: No simulation
Park et al[25]24Group 1: VR simulationGroup 1 performed better: Higher global ratings (17.9 vs 14.8 out of 35) based on technique, use of controls, flow of procedures, advancement.
Group 2: No simulation
Yi et al[26]11Group 1: VR simulationGroup 1 performed better: Higher scores during colonoscopy. Higher number of procedures completed independently (76% vs 43%), higher patient comfort; no difference in time or visualization of mucosa
Group 2: No simulation
Haycock et al[27]36Group 1: VR simulationGroup 1 performed better: Higher completion rates (95% vs 70%) and shorter times (407 s vs 743 s), higher patient comfort, higher use of correction abdominal pressure (79% vs 52%), lower insertion force; other variables such as number of transverse loops, correct use of variable stiffness did not differ
Group 2: Conventional teaching
McIntosh et al[28]18Group 1: VR simulationGroup 1 performed better: Less instances of requiring assistance (1.94 vs 3.43), greater unassisted insertion depth (43 cm vs 24 cm), greater rate of cecal intubation (26% vs 10%), high overall competence scores; patient comfort did not differ
Group 2: No simulation
Gomez et al[29]27Group 1: VR simulation + benchtop simulationGroup 1 and 2 improved: Performed better on post-test compared to pre-test through Global Assessment of Gastrointestinal Endoscopic Skills tool (navigation, strategies, clear lumen and quality of examination)
Group 2: VR simulation
Group 3: Benchtop simulation
Grover et al[30]33Group 1: Self-regulated learning with VR simulationGroup 1 and 2 improved; Group 2 performed better: Both groups improved on colonoscopy-specific performance; Group 2 performed better based on Joint Advisory Group on GI Endoscopy’s Direct Observation of Procedural Skills Tool (JAG DOPS), had better communication rating, and better integrated global rating
Group 2: Structured curriculum with VR simulation
Grover et al[31]37Group 1: Progressive learning with VR simulationGroup 1 performed better: Higher JAG DOPS score, communication and integrated global rating
Group 2: Non-progressive learning with benchtop simulator
Esophagogastroduodenoscopy
Di Giulio et al[32]22Group 1: VR simulationGroup 1 performed better: Higher number of completed procedures (87.8% vs 70%), required less assistance (41.3% vs 97.9%), overall performance was better; length of time was not significantly different
Group 2: No simulation
Sedlack et al[33]8Group 1: VR simulationGroup 1 performed worse: Lower patient comfort (5 vs 6), independence and competence scores
Group 2: No simulation
Shirai et al[34]20Group 1: VR simulation + Conventional teachingGroup 1 performed better: Required less direct assistance (8.6% vs 25.9%), higher score on 11 items scored during the procedure; no significant difference in completion time
Group 2: Conventional teaching
Ferlitsch et al[35]28Group 1: VR simulationGroup 1 performed better: Decreased total time to reach duodenum (239 s vs 310 s); higher technical accuracy; diagnostic accuracy did not differ
Group 2: No simulation
Ende et al[36]29Group 1: VR simulation + Conventional teachingGroup 1 and 2 improved: Improvement in time within group (195 s vs 119 s; 261 s vs 150 s); no significant difference in between groups
Group 2: Conventional teachingAll groups showed improvement in post-intervention manual skills test score.
Group 3: VR simulation aloneNone of the other outcomes reached statistical significance, such as time to intubate esophagus
Endoscopic retrograde cholangiopancreatography (ERCP)
Lim et al[37]16Group 1: Mechanical simulatorGroup 1 performed better: Improved cannulation rates (47.1% vs 69.6%), decreased total time (4.7 vs 10.3 mins); overall performance score not significantly different.
Group 2: No simulator
Liao et al[38]16Group 1: Mechanical simulatorGroup 1 performed better: Improved cannulation rates (73.25% vs 47.35%) and improve overall performance; benefit of single vs multiple simulator practices was not statistically significant.
Group 2: No simulator
Meng et al[39]5Group 1: Mechanical simulatorGroup 1 performed better: Improved cannulation rates (79.4% vs 61.5%), lower total time (19.38 min vs 26.31 min), and improved overall performance score.
Group 2: No simulator
SBT vs apprenticeship model

18 studies investigated SBT and compared it to either no training or conventional model of learning. 16 of the RCTs reported positive results for SBT, i.e., improved outcomes. Of note is that in the 16 RCTs that reported positive results, SBT was either used in conjunction with conventional teaching, or the control arm did not receive any conventional teaching before the assessment period. A closer look at the two negative trials by Gerson and Sedlack reveal several reasons that can explain the results[19,20,33]. In the trial by Gerson and Dam, the SBT group had unlimited access for two weeks to didactic modules and six simulated OGD cases on a VR simulator, with no external feedback or observation, while the conventional training group completed 10 OGDs with supervision from staff endoscopists[20]. As a result, the SBT group completed fewer procedures independently compared to the conventional training group. In the trial by Sedlack, there is no clear explanation for why the SBT group did worse, however the small sample size of 8 trainees and lack of feedback must be taken into consideration[33]. In other trials by Sedlack, trainees who received VR training in combination with conventional training achieved better overall ratings of competency and patient comfort[21,22]. These studies suggest that SBT may have a role in supplementing early endoscopy training. There is, however, no strong evidence to suggest the use of SBT as a replacement for conventional training, giving the limitation of a model in mimicking real life variables and complexities. Furthermore, there is no literature comparing SBT to apprenticeship training for ERCP or other advanced procedures.

SBT with feedback vs without feedback

Feedback is essential in SBT. Recently, Mahmood and Darzi showed that without feedback on performance, SBT does not augment learning[40]. Specifically, the delivery of feedback should be given with an awareness of the trainee’s cognitive load[41]. Cognitive load is the effort used in working memory and has three types: intrinsic, extraneous, and germane[41,42]. Intrinsic load relates to essential components of the learning task, i.e., endoscopy, while extrinsic load relates to non-essential tasks, such as distractions related to other staff members[42]. Germane load relates to forming learning schemas to consolidate the learning, i.e., compartmentalizing concepts to gain mastery[42]. Learning is thought to occur when trainees form their own cognitive schemas that they can readily access[42]. In this setting, feedback can help the trainee through negotiating goals for the current session and setting goals for future sessions. Feedback delivered during endoscopic procedures should be minimal to reduce a trainee’s cognitive load and should be directive. For example, the expert endoscopist may tell or show the trainee how to change their hand position while navigating the endoscope during the session[41]. This allows the trainee to incorporate that feedback during that endoscopy and learn from the mistake. After the session, facilitative feedback, which elicits trainee’s thoughts should be used to help the trainee engage in reflective learning and develop problem solving skills[41]. This is supported by previous work suggesting that delayed feedback, when compared with immediate feedback, may be advantageous in the development of cognitive schemas[41,43].

Integration of SBT in curriculum

Another area of interest in endoscopic training is whether SBT should be embedded in structured curriculum or self-directed curriculum. The benefit of self-directed curriculum is that it allows learners to set their own learning goals and pace their learning[44]. However, this is usually best suited for knowledge sharing that is meant to be sustained for a lifetime or for knowledge that is meant to be applied in varied situations after being acquired in one setting - for example, medical students learning pathophysiology that they will be expected to apply on clinical rotations[45]. Structured learning has a set goal for the trainee, and a trainer to help accomplish that goal[44]. This goal can be in a form of a certain skill or concept. Structured learning works best for teaching cognitive concepts and technical skills. Grover et al[30] investigated SBT as part of a structured curriculum compared to self-regulated curriculum, and found that structured curriculum led to better acquisition of technical and non-technical skills that transferred to the clinical setting. The authors concluded that when using SBT to augment the conventional teaching model, it should be through a structured curriculum that encompasses the cognitive, technical, and integrative skills needed for endoscopy.

Structured curricula can be further enhanced by incorporating educational theory-based interventions. For example, two groups applied progressive learning to endoscopy curricula, which involved challenging the learner by increasing the task difficulty and/or complexity as the learner’s abilities improves[29,31]. In both studies, the authors found that training regimens in which trainees encountered progressively more difficult cases led to improved technical skills in the trainees.

FUTURE DIRECTIONS

Simulation based training through virtual-reality modalities has a role in training the novice endoscopist with no or minimal prior experience in endoscopy. SBT offers the opportunity to practice endoscopy in a risk-free environment prior to first patient contact. It is, however, necessary for trainees to undergo patient-based training with an expert endoscopist. Despite the high fidelity of simulators, it difficult to replicate all the variables that a trainee would encounter during a real patient experience, including the non-technical aspects of endoscopy that need to be mastered. As medical education moves towards a competency-based framework, one area that requires elucidation is level of expertise or competency required in the simulated setting before to moving onto a real patient. Currently, this distinction is unclear and merits further evaluation. Moreover, the use of educational constructs in SBT should be further explored. Furthermore, there is no data in the literature to date on the cost-effectiveness of VR simulation in endoscopy, which can be expensive in terms of capital costs and costs associated with physician-trainers.

Feedback

During SBT, feedback is often provided by an expert that provides through verbal cues and instructions, and/or by performance metrics from the VR simulator. There are several ways to enhance the delivery of feedback. First, before each simulation session, trainees can be encouraged to document and share their objectives with the expert endoscopist, helping them form a habit of being conscious of their goals during each endoscopic procedure. This enhances learning as it provides the trainer and trainee with a specific goal to comment upon[41]. Moreover, taking the feedback from one session, the trainees can be encouraged to reflect upon what they will change in their next session, allowing them to understand how to integrate feedback into their training[41]. SBT in procedural settings allows trainees to observe how incorporating feedback into their next session can result in better outcomes, as other extraneous variables will be controlled. Feedback can be incorporated into endoscopy curricula through gamification, where similar to a game, incremental increase in difficulty and ability to achieve certain goalposts will indicate the level of skill in a trainee[46]. Second, SBT can be used to help identify trainees that are struggling early in their training and allow programs to supplement their learning. Finally, SBT allows for video-recording of procedures and for trainees to watch videos of both their own performance and expert performance. In endoscopy, watching these videos is associated with improved trainee self-assessment skills over time[47].

Non-technical skills

Non-technical components of the endoscopic procedure include communication and teamwork, judgement and decision making, leadership, and situational awareness[48]. Non-technical skills are difficult to teach, especially in a setting where the patient may be partially sedated or even awake. Soft skills in medicine are often taught through role modeling, i.e., apprenticeship[49]. Similarly, non-technical skills in endoscopy are often taught in the endoscopy suite. However, using high fidelity simulation with standardized patients and actors playing the role of nursing staff to simulate integration scenarios can be used help prime trainees to the non-technical skills of endoscopy[30].

Emergencies during endoscopy

Dealing with emergencies during endoscopy can be a very stressful experience for trainees. Currently, SBT is limited to polypectomies and other routine procedures, though it may be expanded to include more emergent cases. Similar to Code Blue training, Kiesslich et al[50] showed that simulation training was associated with better endoscopic performance and crisis management in endoscopic emergencies. Additionally, randomized trials in Advanced Cardiac Life Support simulation revealed improved performance in the simulator-trained groups[51]. SBT allows for trainees to practice the shared mental model and crisis resource management[50]. Further studies randomized trials can explore these concepts.

Ergonomics

Recently, the importance of ergonomics in endoscopy has become evident. Studies have indicated a high prevalence of musculoskeletal disorders ranging from 37%-89% among endoscopists[52]. These injuries are thought to be due to repeated pinching or gripping, pushing, pulling and torquing of the endoscope in potentially awkward postures[53]. Currently, there are no guidelines for optimal ergonomic position due to individual differences in anthropometry. Rather than strict guidelines about positioning during procedures and endoscope handling, some general guidelines may be more valuable. For instance, for computer workstations, guidelines give users recommendations such as keeping the elbow at a right angle with the forearm, which should be parallel to the ground[54]. In endoscopy, posture while doing the procedure should be neutral without excessive bending. Trainees often tend to bend their body unconsciously when trying to get a better view on the screen. Other tips include to use the endoscope as a lever to reduce right hand torqueing, using a neutral thumb grip position, using gauze with right-hand grip to reduce pressure, and using the left-hand pinky grip technique to reduce right-hand strain in difficult or tight endoscope positions[55]. SBT can be used to teach these skills to the trainees by helping them practice posturing with a model that mimics the physical aspects of the endoscope workstation. Studies assessing the effectiveness of SBT to teach ergonomics are lacking.

CONCLUSION

SBT is an important supplement to conventional training to help facilitate learning of endoscopy with no risk to patients. There is, however, insufficient evidence to advocate for SBT as a replacement for conventional training due to its limitation in mimicking real life. Current simulation training curricula may be enhanced with additional focus on the delivery of feedback and the integration of educational theory based strategies. Future studies should broaden the context in which SBT is studied or utilized.

Footnotes

Manuscript source: Invited manuscript

Specialty type: Gastroenterology and hepatology

Country of origin: Canada

Peer-review report classification

Grade A (Excellent): A

Grade B (Very good): B

Grade C (Good): 0

Grade D (Fair): 0

Grade E (Poor): 0

P- Reviewer: Eleftheriadis NP, Hosoe N S- Editor: Wang XJ L- Editor: A E- Editor: Yin SY

References
1.  Ziv A, Ben-David S, Ziv M. Simulation based medical education: an opportunity to learn from errors. Med Teach. 2005;27:193-199.  [PubMed]  [DOI]
2.   Available from: https://www.learning-theories.org/doku.php?id=instructional_design:simulation-based_learning.  [PubMed]  [DOI]
3.  Al-Elq AHSimulation-based medical teaching and learning. J Family Community Med. 2010;17:35-40.  [PubMed]  [DOI]
4.  Cook DA, Hatala R, Brydges R, Zendejas B, Szostek JH, Wang AT, Erwin PJ, Hamstra SJ. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA. 2011;306:978-988.  [PubMed]  [DOI]
5.  Walsh CM, Sherlock ME, Ling SC, Carnahan H. Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev. 2012;CD008237.  [PubMed]  [DOI]
6.  Triantafyllou K, Lazaridis LD, Dimitriadis GD. Virtual reality simulators for gastrointestinal endoscopy training. World J Gastrointest Endosc. 2014;6:6-12.  [PubMed]  [DOI]
7.   Available from: https://www.learning-theories.org/doku.php?id=instructional_design:cognitive_apprenticeship.  [PubMed]  [DOI]
8.  Rassie K. The apprenticeship model of clinical medical education: time for structural change. N Z Med J. 2017;130:66-72.  [PubMed]  [DOI]
9.  Ataizi M Situated Learning. In: Seel NM. Encyclopedia of the Sciences of Learning. Boston, Springer 2012; .  [PubMed]  [DOI]
10.  Bini EJ, Firoozi B, Choung RJ, Ali EM, Osman M, Weinshel EH. Systematic evaluation of complications related to endoscopy in a training setting: A prospective 30-day outcomes study. Gastrointest Endosc. 2003;57:8-16.  [PubMed]  [DOI]
11.  Matharoo M, Haycock A, Sevdalis N, Thomas-Gibson S. A prospective study of patient safety incidents in gastrointestinal endoscopy. Endosc Int Open. 2017;5:E83-E89.  [PubMed]  [DOI]
12.  Dunkin BJ. Flexible endoscopy simulators. Semin Laparosc Surg. 2003;10:29-35.  [PubMed]  [DOI]
13.  Sedlack REThe state of simulation in endoscopy education: continuing to advance toward our goals. Gastroenterology. 2013;144:9-12.  [PubMed]  [DOI]
14.  Kim JS, Kim BW. Training in Endoscopy: Esophagogastroduodenoscopy. Clin Endosc. 2017;50:318-321.  [PubMed]  [DOI]
15.  Desilets DJ, Banerjee S, Barth BA, Kaul V, Kethu SR, Pedrosa MC, Pfau PR, Tokar JL, Varadarajulu S, Wang A. Endoscopic simulators. Gastrointest Endosc. 2011;73:861-867.  [PubMed]  [DOI]
16.  Riegler A. Constructivism. L‘Abate L. Paradigms in theory construction. New York: Springer 2012; 235-256.  [PubMed]  [DOI]
17.  Singh S, Sedlack RE, Cook DA. Effects of simulation-based training in gastrointestinal endoscopy: a systematic review and meta-analysis. Clin Gastroenterol Hepatol. 2014;12:1611-1623.e4.  [PubMed]  [DOI]
18.  Khan R, Plahouras J, Johnston BC, Scaffidi MA, Grover SC, Walsh CM. Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev. 2018;8:CD008237.  [PubMed]  [DOI]
19.  Tuggy ML. Virtual reality flexible sigmoidoscopy simulator training: impact on resident performance. J Am Board Fam Pract. 1998;11:426-433.  [PubMed]  [DOI]
20.  Gerson LB, Van Dam J. A prospective randomized trial comparing a virtual reality simulator to bedside teaching for training in sigmoidoscopy. Endoscopy. 2003;35:569-575.  [PubMed]  [DOI]
21.  Sedlack RE, Kolars JC, Alexander JA. Computer simulation training enhances patient comfort during endoscopy. Clin Gastroenterol Hepatol. 2004;2:348-352.  [PubMed]  [DOI]
22.  Sedlack RE, Kolars JC. Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study. Am J Gastroenterol. 2004;99:33-37.  [PubMed]  [DOI]
23.  Ahlberg G, Hultcrantz R, Jaramillo E, Lindblom A, Arvidsson D. Virtual reality colonoscopy simulation: A compulsory practice for the future colonoscopist? Endoscopy. 2005;37:1198-204.  [PubMed]  [DOI]
24.  Cohen J, Cohen SA, Vora KC, Xue X, Burdick JS, Bank S, Bini EJ, Bodenheimer H, Cerulli M, Gerdes H. Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc. 2006;64:361-368.  [PubMed]  [DOI]
25.  Park J, MacRae H, Musselman LJ, Rossos P, Hamstra SJ, Wolman S, Reznick RK. Randomized controlled trial of virtual reality simulator training: transfer to live patients. Am J Surg. 2007;194:205-211.  [PubMed]  [DOI]
26.  Yi SY, Ryu KH, Na YJ, Woo HS, Ahn W, Kim WS, Lee DY. Improvement of colonoscopy skills through simulation-based training. Stud Health Technol Inform. 2008;132:565-567.  [PubMed]  [DOI]
27.  Haycock A, Koch AD, Familiari P, van Delft F, Dekker E, Petruzziello L, Haringsma J, Thomas-Gibson S. Training and transfer of colonoscopy skills: a multinational, randomized, blinded, controlled trial of simulator versus bedside training. Gastrointest Endosc. 2010;71:298-307.  [PubMed]  [DOI]
28.  McIntosh KS, Gregor JC, Khanna NV. Computer-based virtual reality colonoscopy simulation improves patient-based colonoscopy performance. Can J Gastroenterol Hepatol. 2014;28:203-206.  [PubMed]  [DOI]
29.  Gomez PP, Willis RE, Van Sickle K. Evaluation of two flexible colonoscopy simulators and transfer of skills into clinical practice. J Surg Educ. 2015;72:220-227.  [PubMed]  [DOI]
30.  Grover SC, Garg A, Scaffidi MA, Yu JJ, Plener IS, Yong E, Cino M, Grantcharov TP, Walsh CM. Impact of a simulation training curriculum on technical and nontechnical skills in colonoscopy: a randomized trial. Gastrointest Endosc. 2015;82:1072-1079.  [PubMed]  [DOI]
31.  Grover SC, Scaffidi MA, Khan R, Garg A, Al-Mazroui A, Alomani T, Yu JJ, Plener IS, Al-Awamy M, Yong EL. Progressive learning in endoscopy simulation training improves clinical performance: a blinded randomized trial. Gastrointest Endosc. 2017;86:881-889.  [PubMed]  [DOI]
32.  Di Giulio E, Fregonese D, Casetti T, Cestari R, Chilovi F, D’Ambra G, Di Matteo G, Ficano L, Delle Fave G. Training with a computer-based simulator achieves basic manual skills required for upper endoscopy: a randomized controlled trial. Gastrointest Endosc. 2004;60:196-200.  [PubMed]  [DOI]
33.  Sedlack REValidation of computer simulation training for esophagogastroduodenoscopy: Pilot study. J Gastroenterol Hepatol. 2007;22:1214-1219.  [PubMed]  [DOI]
34.  Shirai Y, Yoshida T, Shiraishi R, Okamoto T, Nakamura H, Harada T, Nishikawa J, Sakaida I. Prospective randomized study on the use of a computer-based endoscopic simulator for training in esophagogastroduodenoscopy. J Gastroenterol Hepatol. 2008;23:1046-1050.  [PubMed]  [DOI]
35.  Ferlitsch A, Schoefl R, Puespoek A, Miehsler W, Schoeniger-Hekele M, Hofer H, Gangl A, Homoncik M. Effect of virtual endoscopy simulator training on performance of upper gastrointestinal endoscopy in patients: a randomized controlled trial. Endoscopy. 2010;42:1049-1056.  [PubMed]  [DOI]
36.  Ende A, Zopf Y, Konturek P, Naegel A, Hahn EG, Matthes K, Maiss J. Strategies for training in diagnostic upper endoscopy: a prospective, randomized trial. Gastrointest Endosc. 2012;75:254-260.  [PubMed]  [DOI]
37.  Lim BS, Leung JW, Lee J, Yen D, Beckett L, Tancredi D, Leung FW. Effect of ERCP mechanical simulator (EMS) practice on trainees’ ERCP performance in the early learning period: US multicenter randomized controlled trial. Am J Gastroenterol. 2011;106:300-306.  [PubMed]  [DOI]
38.  Liao WC, Leung JW, Wang HP, Chang WH, Chu CH, Lin JT, Wilson RE, Lim BS, Leung FW. Coached practice using ERCP mechanical simulator improves trainees’ ERCP performance: a randomized controlled trial. Endoscopy. 2013;45:799-805.  [PubMed]  [DOI]
39.  Meng W, Leung JW, Yue P, Wang Z, Wang X, Wang H. Practice with ercp mechanical simulator (EMS) improves basic ercp skills of novice surgical trainees. Gastrointest Endosc. 2016;83:AB267-AB268.  [PubMed]  [DOI]
40.  Mahmood T, Darzi A. The learning curve for a colonoscopy simulator in the absence of any feedback: no feedback, no learning. Surg Endosc. 2004;18:1224-1230.  [PubMed]  [DOI]
41.  Dilly CK, Sewell JL. How to Give Feedback During Endoscopy Training. Gastroenterology. 2017;153:632-636.  [PubMed]  [DOI]
42.  Leppink J, van den Heuvel A. The evolution of cognitive load theory and its application to medical education. Perspect Med Educ. 2015;4:119-127.  [PubMed]  [DOI]
43.  Walsh CM, Ling SC, Wang CS, Carnahan H. Concurrent versus terminal feedback: it may be better to wait. Acad Med. 2009;84:S54-S57.  [PubMed]  [DOI]
44.   Available from: https://eimf.eu/structured-learning-vs-self-directed-learning/.  [PubMed]  [DOI]
45.  Hung W, Jonassen DH, Liu R. Problem-Based Learning. Handb Res Educ Commun Technol. 2008;3:485-506.  [PubMed]  [DOI]
46.  Burke B. Gamification 2020: What Is the Future of Gamification?  Available from: https://www.gartner.com/doc/2226015/gamification--future-gamification.  [PubMed]  [DOI]
47.  Khan R, Scaffidi MA, Parker C, Al-Mazroui A, Tsui C, Iqbal S. Sa1089 The Influence of Video-Based Feedback on Self-Assessment Accuracy of Procedural Skills. Gastrointest Endosc. 2017;85:AB186.  [PubMed]  [DOI]
48.  Haycock A. Framework for observing and rating Endoscopic Non-Technical Skills.  Available from: https://www.thejag.org.uk/downloads/JAG training information/ENTS handbook v2.pdf.  [PubMed]  [DOI]
49.  Dyche LInterpersonal skill in medicine: the essential partner of verbal communication. J Gen Intern Med. 2007;22:1035-1039.  [PubMed]  [DOI]
50.  Kiesslich R, Moenk S, Reinhardt K, Kanzler S, Schilling D, Jakobs R, Denzer U, Neumann M, Vollmer J, Schütz M. [Combined simulation training: a new concept and workshop is useful for crisis management in gastrointestinal endoscopy]. Z Gastroenterol. 2005;43:1031-1039.  [PubMed]  [DOI]
51.  Sahu S, Lata I. Simulation in resuscitation teaching and training, an evidence based practice review. J Emerg Trauma Shock. 2010;3:378-384.  [PubMed]  [DOI]
52.  Shergill AK, McQuaid KR, Rempel D. Ergonomics and GI endoscopy. Gastrointest Endosc. 2009;70:145-153.  [PubMed]  [DOI]
53.  ASGE Technology CommitteePedrosa MC, Farraye FA, Shergill AK, Banerjee S, Desilets D, Diehl DL, Kaul V, Kwon RS, Mamula P, Rodriguez SA, Varadarajulu S, Song LM, Tierney WM. Minimizing occupational hazards in endoscopy: personal protective equipment, radiation safety, and ergonomics. Gastrointest Endosc. 2010;72:227-235.  [PubMed]  [DOI]
54.  Wahlström JErgonomics, musculoskeletal disorders and computer work. Occup Med (Lond). 2005;55:168-176.  [PubMed]  [DOI]
55.  Chang MA, Mitchell J, Abbas Fehmi SM. Optimizing ergonomics during endoscopy. VideoGIE. 2017;2:170.  [PubMed]  [DOI]