Minireviews
Copyright ©The Author(s) 2019. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastrointest Endosc. Mar 16, 2019; 11(3): 209-218
Published online Mar 16, 2019. doi: 10.4253/wjge.v11.i3.209
Simulation in endoscopy: Practical educational strategies to improve learning
Rishad Khan, Michael A Scaffidi, Samir C Grover, Nikko Gimpaya, Catharine M Walsh
Rishad Khan, Department of Medicine, Schulich School of Medicine and Dentistry, Western University, London ON N6A 5C1, Canada
Rishad Khan, Michael A Scaffidi, Samir C Grover, Nikko Gimpaya, Division of Gastroenterology, St. Michael’s Hospital, University of Toronto, Toronto ON M5B 1W8, Canada
Rishad Khan, Michael A Scaffidi, Samir C Grover, Nikko Gimpaya, Department of Medicine, University of Toronto, Toronto ON M5G 2C4, Canada
Michael A Scaffidi, Faculty of Health Sciences, School of Medicine, Queen’s University, Kingston ON K7L 3N6, Canada
Catharine M Walsh, Division of Gastroenterology, Hepatology, and Nutrition and the Research and Learning Institutes, Hospital for Sick Children, University of Toronto, Toronto ON M5G 1X8, Canada
Catharine M Walsh, Department of Paediatrics, Faculty of Medicine, University of Toronto, Toronto ON M5G 1X8, Canada
Catharine M Walsh, The Wilson Centre, Faculty of Medicine, University of Toronto, Toronto ON M5G 2C4, Canada
ORCID number: Rishad Khan (0000-0002-5090-7685); Michael A Scaffidi (0000-0003-2068-6655); Samir C Grover (0000-0003-3392-1220); Nikko Gimpaya (0000-0003-1015-4372); Catharine M Walsh (0000-0003-3928-703X).
Author contributions: All authors contributed to design and planning, critical revision of manuscript for important intellectual content and approval of final version of manuscript; Khan R and Walsh CM contributed to drafting of the manuscript.
Conflict-of-interest statement: Rishad Khan has received research funding from AbbVie, Ferring Pharmaceuticals, and Pendopharm. Samir C Grover has received research funding from AbbVie and Janssen and personal fees from AbbVie, Takeda, and Ferring, and is owner, and holds shares, in Volō Healthcare. All other authors have no conflicts of interest to disclose.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Corresponding author: Catharine M Walsh, FRCPC, MD, PhD, Clinician Scientist, Division of Gastroenterology, Hepatology, and Nutrition and the Research and Learning Institutes, Hospital for Sick Children, University of Toronto, SickKids PGCRL, 686 Bay St., Room 11.9719, Toronto M5G 1X8, ON, Canada. catharine.walsh@utoronto.ca
Telephone: +1-416-813-7654 x309432
Received: February 12, 2019
Peer-review started: February 14, 2019
First decision: February 26, 2019
Revised: March 6, 2019
Accepted: March 11, 2019
Article in press: March 11, 2019
Published online: March 16, 2019

Abstract

In gastrointestinal endoscopy, simulation-based training can help endoscopists acquire new skills and accelerate the learning curve. Simulation creates an ideal environment for trainees, where they can practice specific skills, perform cases at their own pace, and make mistakes with no risk to patients. Educators also benefit from the use of simulators, as they can structure training according to learner needs and focus solely on the trainee. Not all simulation-based training, however, is effective. To maximize benefits from this instructional modality, educators must be conscious of learners’ needs, the potential benefits of training, and associated costs. Simulation should be integrated into training in a manner that is grounded in educational theory and empirical data. In this review, we focus on four best practices in simulation-based education: deliberate practice with mastery learning, feedback and debriefing, contextual learning, and innovative educational strategies. For each topic, we provide definitions, supporting evidence, and practical tips for implementation.

Key Words: Simulation, Endoscopy, Gastrointestinal, Education

Core tip: In gastrointestinal endoscopy, simulation-based training has been shown to improve learning outcomes and performance in the clinical setting and offers unique advantages to trainees and educators. Four best practices, which are grounded in evidence and can help maximize the learning benefits of simulation-based training, are deliberate practice with mastery learning, feedback and debriefing, contextual learning, and innovative educational strategies.



INTRODUCTION

Simulation-based training allows learners to acquire knowledge, skills and behaviors in a low-risk environment[1]. In gastrointestinal endoscopy, current evidence supports the use of simulation-based training for novice endoscopists to promote skill acquisition, improve performance of initial clinical procedures, and accelerate the learning curve[2-6]. Additionally, simulation can be used to enhance endoscopic non-technical skills and train advanced endoscopic procedures, such as polypectomy[7-9]. Simulation offers an ideal environment for training, as individuals can engage in sustained deliberate practice, work through tasks at their own pace, and build a basic framework of skills and techniques. Importantly, trainees can make mistakes with no patient risk and learn from those mistakes[10]. Simulation also offers advantages for educators, as they can systematically vary training tasks to enhance learning and focus solely on the learner rather than juggle teaching and clinical roles[11].

Despite these advantages, simulation-based training is not universally effective. For example, a 2004 study showed that simulation has no effect on endoscopic skill-acquisition when delivered without feedback from instructors[12]. Simulation is an educational platform through which endoscopy training can be delivered to achieve specific, pre-defined learning goals[13]. Simply providing trainees with access to simulators does not guarantee learning. To be effective, simulation must be integrated into training in a thoughtful and purposeful manner. Additionally, the rationale for incorporating simulation into curricula depends on the magnitude of training benefits, potential cost savings from accelerated learning, and training needs[14]. Overall, the integration of simulation-based training should be thoughtful, deliberate, and grounded in evidence to maximize its learning benefits and outweigh associated costs[15].

This review focuses on four best practices in simulation-based education which can be used to enhance endoscopic training using simulators: (1) deliberate practice with mastery learning; (2) feedback and debriefing; (3) contextual learning; and (4) innovative educational strategies. Within these topics, we will discuss the empirical data supporting their use and practical tips for implementation (Table 1). The benefits of simulation-based training in endoscopy and details of specific endoscopic simulators and curricula have been summarized in multiple recent systematic reviews, and will not be reviewed in depth[2-6]. Additionally, as there is a lack of data on costs associated with endoscopic simulation, this topic will not be covered in this review[16].

Table 1 Best practices in endoscopic simulation-based training.
Educational StrategyKey points
Deliberate practice with mastery learningDeliberate practice: repetitive performance of a skill, constructive feedback, and exercises to correct errors and improve performance
Mastery learning: consistently demonstrating a predefined level of proficiency in a task. Key principles include: baseline assessment; clear and progressive learning objectives; minimum passing standards; educational activities based on predefined objectives and standards; and serial formative assessments to gauge progress
Feedback and debriefingEndoscopic simulation in the absence of feedback may be ineffective
Feedback should be simple, goal-directed, based on observable behaviors, and ideally delivered during a debrief at the end of a simulated procedure
Educators may supplement feedback with validated endoscopic assessment tools and input from other sources, such as nurses, anesthesiologists, and standardized patients
Debriefing should be a two-way process through which trainees and their trainers identify gaps in performance, explore the basis of those gaps, and establish tasks to improve performance
Contextual learningInitial training should focus on acquisition of basic skills such as endoscope navigation and torque steering, and progress to simulated tasks of increasing complexity and difficulty
The introduction of team-based practice through hybrid simulation models can allow trainees to practice non-technical skills, such as communication, decision making, leadership, and crisis management
Varying tasks during training can better prepare trainees to handle variation in anatomy, pathology, and difficulty during real procedures
Innovative educational designEndoscopy simulation curricula grounded in educational theory and empirical data have been shown to improve transfer of learning outcomes to the clinical environment
Training programs can improve learning by implementing simulation sessions at more widely spaced intervals
Just-in-time simulation training may be used to allow trainees to “warm-up” before performing complex tasks in the clinical environment
Novel educational strategies emerging in simulation include the application of game design elements and the use of head-mounted displays to create an immersive experience
DELIBERATE PRACTICE WITH MASTERY LEARNING

Not all practice is perfect. Practice must be purposeful and systematic or “deliberate”. Deliberate practice involves focused repetitive performance of a skill, coupled with constructive feedback that identifies weaknesses, and promotes self-reflection and error correction to improve performance[17]. Simulation-based training should be delivered in such a way that it allows learners to practice important skills, receive focused feedback, and improve until they achieve mastery. Mastery refers to the ability to consistently demonstrate a predefined level of proficiency on a task before advancing to the next task[18,19]. In this way, individuals progress through tasks of increasing level of difficulty. Key principles in mastery-learning models include a baseline assessment to determine the appropriate level of difficulty of initial simulation-based activities, clear and progressive learning objectives, minimum passing standards (i.e., learning outcomes), educational activities focused on achieving predefined objectives and standards, and serial formative assessments to gauge progress[19,20]. For mastery learning to be most effective there should be multiple different simulation experiences which increase in challenge.

In a recent systematic review of studies in procedural settings, such as surgery and airway management, simulation-based training with mastery learning was associated with better learning outcomes as compared to training without[18]. Additionally, randomized trials in resuscitation and laparoscopic surgery have shown that deliberate practice-based models lead to superior performance in both the clinical and simulated settings[21-24]. In endoscopy, no studies exist which directly compare mastery learning or deliberate practice with other simulation-based learning strategies. One study, however, found that a mastery learning-based simulation curriculum, as compared with no training, resulted in superior clinical colonoscopy performance[25]. Two other pre-post studies found that mastery learning-based curricula resulted in improved performance of simulated colonoscopy[26,27].

Simulation offers an ideal setting for trainees to engage in mastery learning principles and deliberate practice without posing risk to patients[28]. The simulated environment allows learners to repetitively perform the intended skills, receive focused feedback to identify and correct errors, and adjust training to target specific skills or build upon existing competencies with increasing levels of challenge[17]. Despite these potential advantages, incorporating mastery learning principles poses several challenges. First, as trainees are required to all meet the same objectives, training time will vary. In many cases, a mastery model will require more time[18]. Additionally, learning objectives, key simulation-based metrics and minimum passing standards in endoscopy are not well defined.

FEEDBACK AND DEBRIEFING

Provision of data on a performance (feedback) and conversations about the performance (debriefing) drive improvement and are essential components of simulation-based training[29,30]. Endoscopic simulation in the absence of these elements may be ineffective[2,12]. Additionally, a recent randomized trial demonstrated that a structured, simulation-based curriculum which included feedback and debriefing with expert endoscopists, led to superior transfer of skills to the clinical environment, compared to self-regulated simulation-based training with no feedback or debriefing[31]. Given the importance of these practices, it is important to align feedback and debriefing with the goals of endoscopic training. Practical considerations include the timing of feedback, the content, and the manner in which feedback is delivered.

In the simulated setting, trainees can progress through cases and solve problems independently with no risk to patients. This allows learners to receive feedback after completion of a procedure, a practice that is more effective for endoscopic skill-acquisition compared to feedback received during a procedure[32]. Constant feedback may place an increase cognitive load on novice endoscopists as they attempt to focus on both the procedure and their instructors’ feedback[33]. Additionally, trainees may begin to rely on feedback as instruction to guide them through procedures and the skills is not optimally learned[34]. Feedback during a procedure should be limited to providing key information when required. Additionally, when receiving feedback during a procedure, the trainee should be asked to briefly stop what they are doing so they can focus on the feedback and then proceed with the procedure. Delivery of feedback during a post-procedure debriefing session is key as it allows the trainee and trainer to mutually identify gaps in training, explore the basis of the gaps, and set activities for skills improvement[35].

In keeping with the principles of mastery learning and deliberate practice, feedback should be specific, goal-directed, actionable, and focused on improvement[17,36,37]. Feedback should be non-judgmental, relate to pre-specified objectives, it should be based on observable behaviors and it should focus on well-defined and achievable points to avoid overburdening the trainee. Engaging trainees in a two-way feedback conversation is crucial, as it helps to promote self-reflection. Feedback should aim to foster trainee’s conscious understanding of the procedure. As trainees advance, the feedback conversation should focus on critical challenges that arose during the simulated procedure, encourage the learner to reflect on the problem and propose potential solutions which can be then be discussed[11]. Questioning encourages active engagement, reflection and independent thought rather than simply being informed of the best option[37].

Trainers can supplement feedback discussions with objective indicators of performance such as a video of the simulated procedure or data from endoscopy assessment tools with strong validity evidence. These tools, which include the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT)[38], the Mayo Colonoscopy Skills Assessment Tool (MCSAT)[39], the Assessment of Competency in Endoscopy (ACE) tool[40], and the Joint Advisory Committee of GI Endoscopy’s Direct Observation of Procedure (JAG DOPS) Assessment Tool[41], can help guide debrief sessions and identify areas of weakness. Feedback from other sources can add another dimension to simulation-based training sessions and help to further characterize trainees’ deficiencies. For example, the Nurse-Assessed Patient Comfort Score (NAPCOMS) may be employed with high-fidelity simulators where indicators of patient comfort and sedation are available throughout the procedure[42]. Additionally, training programs can implement a hybrid simulation model, in which trainees practice on a simulator while interacting with a standardized patient (actor portraying a patient)[43]. Through these simulated cases, standardized patients and nurses can participate in debriefing and act as additional sources of feedback. They can also provide insight into the integrative and cognitive aspects of endoscopy, in addition to the technical aspects. Proficiency in all three of these domains is required for competence in endoscopy, and thus they are increasingly incorporated into simulation-based curricula in endoscopy and assessment tools[31,44].

With a growing emphasis on patient safety and a shift towards competency-based postgraduate training curricula in gastroenterology, the provision of feedback and debriefing to enhance performance is crucial[45,46]. Using the large body of empirical research on these topics, instructors can help trainees continually build upon their competencies in endoscopy.

CONTEXTUAL LEARNING

A fundamental concept for instructional design of endoscopic simulation-based training curricula is the applicability, or transfer of training experiences to clinical performance. This transfer can be affected by a range of factors related to the context of training, including trainees’ developmental levels, provision of team training, and task variability[30].

Simulation-based training should match specific learning objectives and a learner’s developmental level. For example, novice endoscopists can acquire the basic skills of video interpretation, endoscopic handling, and torque steering by practicing on a low-fidelity, bench-top simulator[47]. Training on low-fidelity simulators allows educators to attach precise tasks with physical platforms to target specific learning objectives, a concept known as functional task alignment[13]. This design approach has been identified as a key feature of effective simulation in multiple systematic reviews[1,20,48]. In a recent randomized trial, learners progressed from a low-fidelity, bench-top simulator to a virtual reality simulator with higher fidelity and completed simulated cases in order of increasing complexity and difficulty (Figure 1)[47]. This progressive model of learning improved skill acquisition and transfer of skills to the clinical setting compared to a curriculum using only high-fidelity simulation, supporting the notion that aligning task difficulty to learner skill allows learners to be optimally challenged, which, ultimately, enhances learning[49].

Figure 1
Figure 1 An example of a progressive model of endoscopic simulation-based training whereby learners complete tasks of progressively increasing difficulty as their skills improve. Endoscopic simulators are matched to the task

Simulation also offers opportunities to train endoscopists in team-based settings using the aforementioned hybrid simulation model[8,43]. In this model, simulators are linked to a simulated patient and other team members, such as an endoscopy nurse or anesthesiologist. Learners can engage in these simulations in the naturalistic setting of an endoscopy suite and perform procedures while building their skills in communication, decision making, leadership, coordination, and crisis management. Practicing in team-based settings can help automate such behaviors, making them more resilient to the effects of stress, which, in turn, leads to improved performance under stressful conditions[50]. Recent randomized trials support the use of hybrid simulation in endoscopy as a means to improve transfer of critical non-technical skills to the clinical environment[7,31,47].

Another important factor in the applicability of training experiences to the clinical environment is task variability. Live endoscopic procedures present variation with respect to anatomy, procedural difficulty, and pathology encountered. Varying tasks during simulation-based training can increase exposure to a broader range of endoscopic skills and situations, and result in enhanced initial skill acquisition and long-term retention of skills[1,51]. While no studies have examined the impact of task variability in endoscopy, a study from the laparoscopic surgery literature suggests that simulation-based training incorporating variability improves flexibility of trained skills among trainees. Endoscopy teachers can incorporate these principles by using a combination of different cases on both low- and high-fidelity simulators, as described above, and incorporating modules to train specific technical skills, such as polypectomy, or cognitive skills, such as lesion recognition[9].

INNOVATIVE EDUCATIONAL DESIGN

Endoscopy curricula are increasingly incorporating instructional design elements grounded in educational theory and empirical findings from the educational literature. Recent studies by Grover et al[7,31,47] have demonstrated the potential benefits of this strategy, with trials of simulation-based training with a structured curriculum, a progressive learning model, and with structured non-technical skills training resulting in improved transfer of skills to the clinical environment. Additionally, there are several emerging educational strategies that can potentially be applied to endoscopic simulation-based training including spaced practice, just-in-time training, gamification, and immersive virtual reality.

In spaced practice, training is separated into several discrete sessions over a prolonged period. In contrast, most endoscopy curricula are delivered as massed practice, with training taking place during a single time period lasting hours or days[2]. Practice distributed over time yields better learning than compressed practice, in a phenomenon known as the spacing effect[52]. While no studies have evaluated spaced and massed practice directly in endoscopy, a recent trial by Ende et al[53] described novice endoscopists performing simulated cases for two hours each week, over a four month period. Trainees who underwent this spaced practice program had superior performance of diagnostic upper endoscopy compared to trainees who practiced on real patients in the 4-mo window[53]. Educators with access to simulators can take advantage of spaced practice principles by introducing booster sessions, which describe training sessions which take place after initial massed training, and just-in-time training, which describe refresher sessions conducted prior to a luminal rotation with a high endoscopic case volume[54-56]. Just-in-time simulation training could also be used to prepare trainees for more complex skills such as polypectomy, whereby trainees ‘warm-up’ on a simulator before completing the task in real life; a strategy which has been shown to be useful in other procedural domains[57,58].

Another innovative and potentially applicable educational strategy is gamification. Gamification, or the application of game design elements (e.g., points, badges, and leaderboards) to a traditionally nongame contexts (e.g., simulation curricula, learning activity), is increasingly being used within medical education[59,60]. Studies from the broader simulation literature highlight the potential role of gamification as a means to enhance leaner motivation, engagement and procedural skills performance[59,61-64]. For example, MacKinnon et al[63] showed that a leaderboard was a positive motivator for simulated CPR practice and Mokadam et al[62] used gamification to increase trainees’ use of a small-vessel anastomosis simulator, resulting in skills improvement. Game design elements which rank participants, such as leaderboards, are purported to increase learners’ sense of control and competence as they enable learners to set attainable process goals[59]. Additionally, gamification can potentially enhance learners’ sense of relatedness (interconnectedness with other learners and teachers) which is thought to enhance engagement[59]. While gamification is a potentially useful educational strategy, there is only one study, which is currently in progress, that aims to examine the use of gamification within the endoscopic simulation-based training context[65]. Educators must remember that when integrating gamification, it must be done so purposefully, in that it should align with the learning goals of the simulation-based training to enhance learner motivation and engagement, and ultimately, improve learning[59].

Recently, the concept of immersive virtual reality has been introduced in simulation research. This represents an attempt to improve the realism of simulated settings and increase the user’s sense of presence. For example, a recent study in laparoscopic surgery reported on the integration of a virtual reality simulator with a head-mounted display to create an immersive experience in which users have a wide field of view with head tracking and depth perception that more closely represents human vision[66]. The use of such displays has received positive reviews from operating room staff and has been shown to improve response time and performance scores during a simulation of an operating room emergency[67,68]. While studies are needed to assess the learning benefits of immersive virtual reality in endoscopy, the rise of commercially available virtual reality head-mounted displays may allow for the incorporation of this technology into simulation training programs.

CONCLUSION

Simulation-based training is increasingly being incorporated into endoscopy curricula. Despite its growing use, there remains a need to integrate evidence-based strategies such as deliberate practice with mastery learning, feedback and debriefing, contextual learning, and innovative educational design. Educators looking to implement simulation-based training should consider the specific objectives of training, learner’s needs, the magnitude of potential training benefits, and associated costs and prospective savings. When done in a thoughtful and deliberate manner, training programs can maximize the potential learning benefits of simulation.

Footnotes

Manuscript source: Invited manuscript

Specialty type: Gastroenterology and hepatology

Country of origin: Canada

Peer-review report classification

Grade A (Excellent): 0

Grade B (Very good): B

Grade C (Good): C

Grade D (Fair): 0

Grade E (Poor): 0

P- Reviewer: Kamimura K, Sandhu DS S- Editor: Ji FF L- Editor: A E- Editor: Zhang YL

References
1.  Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27:10-28.  [PubMed]  [DOI]
2.  Khan R, Plahouras J, Johnston BC, Scaffidi MA, Grover SC, Walsh CM. Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev. 2018;8:CD008237.  [PubMed]  [DOI]
3.  Qiao W, Bai Y, Lv R, Zhang W, Chen Y, Lei S, Zhi F. The effect of virtual endoscopy simulator training on novices: a systematic review. PLoS One. 2014;9:e89224.  [PubMed]  [DOI]
4.  Dawe SR, Windsor JA, Broeders JA, Cregan PC, Hewett PJ, Maddern GJ. A systematic review of surgical skills transfer after simulation-based training: laparoscopic cholecystectomy and endoscopy. Ann Surg. 2014;259:236-248.  [PubMed]  [DOI]
5.  Ekkelenkamp VE, Koch AD, de Man RA, Kuipers EJ. Training and competence assessment in GI endoscopy: a systematic review. Gut. 2016;65:607-615.  [PubMed]  [DOI]
6.  Singh S, Sedlack RE, Cook DA. Effects of simulation-based training in gastrointestinal endoscopy: a systematic review and meta-analysis. Clin Gastroenterol Hepatol. 2014;12:1611-23.e4.  [PubMed]  [DOI]
7.  Grover SC, Scaffidi MA, Khan R, Chana B, Iqbal S, Kalaichandran S, Tsui C, Zasowski M, Al-Mazroui A, Sharma S. Sa1075 A Virtual Reality Curriculum in Non-Technical Skills Improves Colonoscopic Performance: A Randomized Trial. Gastrointest Endosc. 2017;85:AB181.  [PubMed]  [DOI]
8.  Khan R, Scaffidi MA, Walsh CM, Lin P, Al-Mazroui A, Chana B, Kalaichandran R, Lee W, Grantcharov TP, Grover SC. Simulation-Based Training of Non-Technical Skills in Colonoscopy: Protocol for a Randomized Controlled Trial. JMIR Res Protoc. 2017;6:e153.  [PubMed]  [DOI]
9.  Gomez PP, Willis RE, Van Sickle K. Evaluation of two flexible colonoscopy simulators and transfer of skills into clinical practice. J Surg Educ. 2015;72:220-227.  [PubMed]  [DOI]
10.  Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an ethical imperative. Acad Med. 2003;78:783-788.  [PubMed]  [DOI]
11.  Walsh CM, Anderson JT, Fishman DS. Evidence-based Approach to Training Pediatric Gastrointestinal Endoscopy Trainers. J Pediatr Gastroenterol Nutr. 2017;64:501-504.  [PubMed]  [DOI]
12.  Mahmood T, Darzi A. The learning curve for a colonoscopy simulator in the absence of any feedback: no feedback, no learning. Surg Endosc. 2004;18:1224-1230.  [PubMed]  [DOI]
13.  Hamstra SJ, Brydges R, Hatala R, Zendejas B, Cook DA. Reconsidering fidelity in simulation-based training. Acad Med. 2014;89:387-392.  [PubMed]  [DOI]
14.  Cohen J, Bosworth BP, Chak A, Dunkin BJ, Early DS, Gerson LB, Hawes RH, Haycock AV, Hochberger JH, Hwang JH, Martin JA, McNally PR, Sedlack RE, Vassiliou MC. Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI) on the use of endoscopy simulators for training and assessing skill. Gastrointest Endosc. 2012;76:471-475.  [PubMed]  [DOI]
15.  Walsh CM, Cohen J. Endoscopic Simulators. In: Chandrasekhara V, Elmunzer BJ, Khashab MA, Muthusamy VR, eds. Clinical Gastrointestinal Endoscopy (Third Edition). Philadelphia, PA: Elsevier 2019; 141-151.  [PubMed]  [DOI]
16.  Zendejas B, Wang AT, Brydges R, Hamstra SJ, Cook DA. Cost: the missing outcome in simulation-based medical education research: a systematic review. Surgery. 2013;153:160-176.  [PubMed]  [DOI]
17.  Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79:S70-S81.  [PubMed]  [DOI]
18.  Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Mastery learning for health professionals using technology-enhanced simulation: a systematic review and meta-analysis. Acad Med. 2013;88:1178-1186.  [PubMed]  [DOI]
19.  McGaghie WC. Mastery learning: it is time for medical education to join the 21st century. Acad Med. 2015;90:1438-1441.  [PubMed]  [DOI]
20.  McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003-2009. Med Educ. 2010;44:50-63.  [PubMed]  [DOI]
21.  Wayne DB, Butter J, Siddall VJ, Fudala MJ, Linquist LA, Feinglass J, Wade LD, McGaghie WC. Simulation-based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial. Teach Learn Med. 2005;17:210-216.  [PubMed]  [DOI]
22.  Andreatta PB, Woodrum DT, Birkmeyer JD, Yellamanchilli RK, Doherty GM, Gauger PG, Minter RM. Laparoscopic skills are improved with LapMentor training: results of a randomized, double-blinded study. Ann Surg. 2006;243:854-860; discussion 860-863.  [PubMed]  [DOI]
23.  Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, Ramel S, Smith CD, Arvidsson D. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg. 2007;193:797-804.  [PubMed]  [DOI]
24.  Korndorffer JR, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg. 2005;201:23-29.  [PubMed]  [DOI]
25.  Ahlberg G, Hultcrantz R, Jaramillo E, Lindblom A, Arvidsson D. Virtual reality colonoscopy simulation: a compulsory practice for the future colonoscopist? Endoscopy. 2005;37:1198-1204.  [PubMed]  [DOI]
26.  Van Sickle KR, Buck L, Willis R, Mangram A, Truitt MS, Shabahang M, Thomas S, Trombetta L, Dunkin B, Scott D. A multicenter, simulation-based skills training collaborative using shared GI Mentor II systems: results from the Texas Association of Surgical Skills Laboratories (TASSL) flexible endoscopy curriculum. Surg Endosc. 2011;25:2980-2986.  [PubMed]  [DOI]
27.  Bistritz LM. Use of a colonoscopy simulator for resident teaching: Application of a mastery learning model. MHPE [dissertation]. Chicago: University of Illinois at Chicago; 2015.  Available from: https://indigo.uic.edu/bitstream/handle/10027/19668/Bistritz_Lana.pdf?sequence=1.  [PubMed]  [DOI]
28.  McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011;86:706-711.  [PubMed]  [DOI]
29.  Motola I, Devine LA, Chung HS, Sullivan JE, Issenberg SB. Simulation in healthcare education: a best evidence practical guide. AMEE Guide No. 82. Med Teach. 2013;35:e1511-e1530.  [PubMed]  [DOI]
30.  Cheng A, Nadkarni VM, Mancini MB, Hunt EA, Sinz EH, Merchant RM, Donoghue A, Duff JP, Eppich W, Auerbach M, Bigham BL, Blewer AL, Chan PS, Bhanji F; American Heart Association Education Science Investigators; and on behalf of the American Heart Association Education Science and Programs Committee, Council on Cardiopulmonary, Critical Care, Perioperative and Resuscitation; Council on Cardiovascular and Stroke Nursing; and Council on Quality of Care and Outcomes Research. Resuscitation Education Science: Educational Strategies to Improve Outcomes From Cardiac Arrest: A Scientific Statement From the American Heart Association. Circulation. 2018;138:e82-e122.  [PubMed]  [DOI]
31.  Grover SC, Garg A, Scaffidi MA, Yu JJ, Plener IS, Yong E, Cino M, Grantcharov TP, Walsh CM. Impact of a simulation training curriculum on technical and nontechnical skills in colonoscopy: a randomized trial. Gastrointest Endosc. 2015;82:1072-1079.  [PubMed]  [DOI]
32.  Walsh CM, Ling SC, Wang CS, Carnahan H. Concurrent versus terminal feedback: it may be better to wait. Acad Med. 2009;84:S54-S57.  [PubMed]  [DOI]
33.  van Merriënboer JJ, Sweller J. Cognitive load theory in health professional education: design principles and strategies. Med Educ. 2010;44:85-93.  [PubMed]  [DOI]
34.  Salmoni AW, Schmidt RA, Walter CB. Knowledge of results and motor learning: a review and critical reappraisal. Psychol Bull. 1984;95:355-386.  [PubMed]  [DOI]
35.  Rudolph JW, Simon R, Raemer DB, Eppich WJ. Debriefing as formative assessment: closing performance gaps in medical education. Acad Emerg Med. 2008;15:1010-1016.  [PubMed]  [DOI]
36.  Fried GM, Waschke KA.  How endoscopy is learned: deconstructing skill sets. In: Successful Training in Gastrointestinal Endoscopy 2011; 16-21.  [PubMed]  [DOI]
37.  Anderson JT, Valori R. Training for Trainers in Endoscopy (Colonoscopy). Training in Minimal Access Surgery. London: Springer 2015; 61-78.  [PubMed]  [DOI]
38.  Walsh CM, Ling SC, Khanna N, Grover SC, Yu JJ, Cooper MA, Yong E, Nguyen GC, May G, Walters TD, Reznick R, Rabeneck L, Carnahan H. Gastrointestinal Endoscopy Competency Assessment Tool: reliability and validity evidence. Gastrointest Endosc. 2015;81:1417-1424.e2.  [PubMed]  [DOI]
39.  Sedlack RE. The Mayo Colonoscopy Skills Assessment Tool: validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc. 2010;72:1125-1133, 1133.e1-1133.e3.  [PubMed]  [DOI]
40.  Sedlack RE, Coyle WJ; ACE Research Group. Assessment of competency in endoscopy: establishing and validating generalizable competency benchmarks for colonoscopy. Gastrointest Endosc. 2016;83:516-23.e1.  [PubMed]  [DOI]
41.  Barton JR, Corbett S, van der Vleuten CP; English Bowel Cancer Screening Programme; UK Joint Advisory Group for Gastrointestinal Endoscopy. The validity and reliability of a Direct Observation of Procedural Skills assessment tool: assessing colonoscopic skills of senior endoscopists. Gastrointest Endosc. 2012;75:591-597.  [PubMed]  [DOI]
42.  Rostom A, Ross ED, Dubé C, Rutter MD, Lee T, Valori R, Bridges RJ, Pontifex D, Webbink V, Rees C, Brown C, Whetter DH, Kelsey SG, Hilsden RJ. Development and validation of a nurse-assessed patient comfort score for colonoscopy. Gastrointest Endosc. 2013;77:255-261.  [PubMed]  [DOI]
43.  Kneebone R, Nestel D, Yadollahi F, Brown R, Nolan C, Durack J, Brenton H, Moulton C, Archer J, Darzi A. Assessing procedural skills in context: Exploring the feasibility of an Integrated Procedural Performance Instrument (IPPI). Med Educ. 2006;40:1105-1114.  [PubMed]  [DOI]
44.  Walsh CM, Ling SC, Khanna N, Cooper MA, Grover SC, May G, Walters TD, Rabeneck L, Reznick R, Carnahan H. Gastrointestinal Endoscopy Competency Assessment Tool: development of a procedure-specific assessment tool for colonoscopy. Gastrointest Endosc. 2014;79:798-807.e5.  [PubMed]  [DOI]
45.  Patel SG, Keswani R, Elta G, Saini S, Menard-Katcher P, Del Valle J, Hosford L, Myers A, Ahnen D, Schoenfeld P, Wani S. Status of Competency-Based Medical Education in Endoscopy Training: A Nationwide Survey of US ACGME-Accredited Gastroenterology Training Programs. Am J Gastroenterol. 2015;110:956-962.  [PubMed]  [DOI]
46.  Bolhari B, Sharifian MR, Aminsobhani M, Monsef Esfehani HR, Tavakolian P. Assessing the efficacy of citrus aurantifolia extract on smear layer removal with scanning electron microscope. Iran Endod J. 2012;7:88-97.  [PubMed]  [DOI]
47.  Grover SC, Scaffidi MA, Khan R, Garg A, Al-Mazroui A, Alomani T, Yu JJ, Plener IS, Al-Awamy M, Yong EL, Cino M, Ravindran NC, Zasowski M, Grantcharov TP, Walsh CM. Progressive learning in endoscopy simulation training improves clinical performance: a blinded randomized trial. Gastrointest Endosc. 2017;86:881-889.  [PubMed]  [DOI]
48.  Cook DA, Hamstra SJ, Brydges R, Zendejas B, Szostek JH, Wang AT, Erwin PJ, Hatala R. Comparative effectiveness of instructional design features in simulation-based education: systematic review and meta-analysis. Med Teach. 2013;35:e867-e898.  [PubMed]  [DOI]
49.  Guadagnoli M, Morin MP, Dubrowski A. The application of the challenge point framework in medical education. Med Educ. 2012;46:447-453.  [PubMed]  [DOI]
50.  Paris CR, Salas E, Cannon-Bowers JA. Teamwork in multi-person systems: a review and analysis. Ergonomics. 2000;43:1052-1075.  [PubMed]  [DOI]
51.  Jordan JA, Gallagher AG, McGuigan J, McGlade K, McClure N. A comparison between randomly alternating imaging, normal laparoscopic imaging, and virtual reality training in laparoscopic psychomotor skill acquisition. Am J Surg. 2000;180:208-211.  [PubMed]  [DOI]
52.  Ebbinghaus H. Memory: a contribution to experimental psychology. Ann Neurosci. 2013;20:155-156.  [PubMed]  [DOI]
53.  Ende A, Zopf Y, Konturek P, Naegel A, Hahn EG, Matthes K, Maiss J. Strategies for training in diagnostic upper endoscopy: a prospective, randomized trial. Gastrointest Endosc. 2012;75:254-260.  [PubMed]  [DOI]
54.  Kessler D, Pusic M, Chang TP, Fein DM, Grossman D, Mehta R, White M, Jang J, Whitfill T, Auerbach M; INSPIRE LP investigators. Impact of Just-in-Time and Just-in-Place Simulation on Intern Success With Infant Lumbar Puncture. Pediatrics. 2015;135:e1237-e1246.  [PubMed]  [DOI]
55.  Niles D, Sutton RM, Donoghue A, Kalsi MS, Roberts K, Boyle L, Nishisaki A, Arbogast KB, Helfaer M, Nadkarni V. "Rolling Refreshers": a novel approach to maintain CPR psychomotor skill competence. Resuscitation. 2009;80:909-912.  [PubMed]  [DOI]
56.  Braga MS, Tyler MD, Rhoads JM, Cacchio MP, Auerbach M, Nishisaki A, Larson RJ. Effect of just-in-time simulation training on provider performance and patient outcomes for clinical procedures: a systematic review. BMJ Simul Technol Enhanc Learn. 2015;1:94–102.  [PubMed]  [DOI]
57.  Samuelson ST, Burnett G, Sim AJ, Hofer I, Weinberg AD, Goldberg A, Chang TS, DeMaria S. Simulation as a set-up for technical proficiency: can a virtual warm-up improve live fibre-optic intubation? Br J Anaesth. 2016;116:398-404.  [PubMed]  [DOI]
58.  Mucksavage P, Lee J, Kerbl DC, Clayman RV, McDougall EM. Preoperative warming up exercises improve laparoscopic operative times in an experienced laparoscopic surgeon. J Endourol. 2012;26:765-768.  [PubMed]  [DOI]
59.  Rutledge C, Walsh CM, Swinger N, Auerbach M, Castro D, Dewan M, Khattab M, Rake A, Harwayne-Gidansky I, Raymond TT, Maa T, Chang TP; Quality Cardiopulmonary Resuscitation (QCPR) leaderboard investigators of the International Network for Simulation-based Pediatric Innovation, Research, and Education (INSPIRE). Gamification in Action: Theoretical and Practical Considerations for Medical Educators. Acad Med. 2018;93:1014-1020.  [PubMed]  [DOI]
60.  Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining gamification. ACM. 2011;2011:9-15.  [PubMed]  [DOI]
61.  Kerfoot BP, Kissane N. The use of gamification to boost residents' engagement in simulation training. JAMA Surg. 2014;149:1208-1209.  [PubMed]  [DOI]
62.  Mokadam NA, Lee R, Vaporciyan AA, Walker JD, Cerfolio RJ, Hermsen JL, Baker CJ, Mark R, Aloia L, Enter DH, Carpenter AJ, Moon MR, Verrier ED, Fann JI. Gamification in thoracic surgical education: Using competition to fuel performance. J Thorac Cardiovasc Surg. 2015;150:1052-1058.  [PubMed]  [DOI]
63.  MacKinnon R, Stoeter R, Doherty C, Fullwood C, Cheng A, Nadkarni V, Stenfors-Hayes T, Chang T. Self-motivated learning with gamification improves infant CPR performance, a randomised controlled trial. BMJ Simul Technol Enhanc Learn. 2015;1:71-76.  [PubMed]  [DOI]
64.  Hashimoto DA, Gomez ED, Beyer-Berjot L, Khajuria A, Williams NN, Darzi A, Aggarwal R. A Randomized Controlled Trial to Assess the Effects of Competition on the Development of Laparoscopic Surgical Skills. J Surg Educ. 2015;72:1077-1084.  [PubMed]  [DOI]
65.  Scaffidi M, Walsh C, Pearl M, Khan R, Kalaichandran R, Lui E, Winger K, Almazroui A, Abunassar M, Grover S. A204 Impact of a simulation-based training curriculum using gamification for colonoscopy: A randomized controlled trial. J Can Assoc Gastroenterol. 2018;1:302–302.  [PubMed]  [DOI]
66.  Huber T, Wunderling T, Paschold M, Lang H, Kneist W, Hansen C. Highly immersive virtual reality laparoscopy simulation: development and future aspects. Int J Comput Assist Radiol Surg. 2018;13:281-290.  [PubMed]  [DOI]
67.  Huber T, Paschold M, Hansen C, Wunderling T, Lang H, Kneist W. New dimensions in surgical training: immersive virtual reality laparoscopic simulation exhilarates surgical staff. Surg Endosc. 2017;31:4472-4477.  [PubMed]  [DOI]
68.  Sankaranarayanan G, Wooley L, Hogg D, Dorozhkin D, Olasky J, Chauhan S, Fleshman JW, De S, Scott D, Jones DB. Immersive virtual reality-based training improves response in a simulated operating room fire scenario. Surg Endosc. 2018;32:3439-3449.  [PubMed]  [DOI]