Topic Highlight
Copyright ©2006 Baishideng Publishing Group Co., Limited. All rights reserved.
World J Gastroenterol. May 28, 2006; 12(20): 3219-3224
Published online May 28, 2006. doi: 10.3748/wjg.v12.i20.3219
Evolution of surgical skills training
Kurt E Roberts, Robert L Bell, Andrew J Duffy
Kurt E Roberts, Robert L Bell, Andrew J Duffy, Department of Surgery, Yale University School of Medicine, 40 Temple Street, Suite 3A, New Haven, CT 06510, United Staets
Author contributions: All authors contributed equally to the work.
Correspondence to: Andrew J Duffy, MD, Department of Surgery, Yale University School of Medicine, 40 Temple Street, Suite 3A, New Haven, CT 06510, United Staets. andrew.duffy@yale.edu
Telephone: +1-203-7469060 Fax: +1-203-7469066
Received: March 29, 2006
Revised: April 6, 2006
Accepted: April 16, 2006
Published online: May 28, 2006

Abstract

Surgical training is changing: one hundred years of tradition is being challenged by legal and ethical concerns for patient safety, work hours restrictions, the cost of operating room time, and complications. Surgical simulation and skills training offers an opportunity to teach and practice advanced skills outside of the operating room environment before attempting them on living patients.

Simulation training can be as straight forward as using real instruments and video equipment to manipulate simulated “tissue” in a box trainer. More advanced, virtual reality simulators are now available and ready for widespread use. Early systems have demonstrated their effectiveness and discriminative ability. Newer systems enable the development of comprehensive curricula and full procedural simulations.

The Accreditation Council of Graduate Medical Education’s (ACGME) has mandated the development of novel methods of training and evaluation. Surgical organizations are calling for methods to ensure the maintenance of skills, advance surgical training, and to credential surgeons as technically competent.

Simulators in their current form have been demonstrated to improve the operating room performance of surgical residents. Development of standardized training curricula remains an urgent and important agenda, particularly for minimal invasive surgery. An innovative and progressive approach, borrowing experiences from the field of aviation, can provide the foundation for the next century of surgical training, ensuring the quality of the product. As the technology develops, the way we practice will continue to evolve, to the benefit of physicians and patients.

Key Words: Surgery, Training, Certification, Skills, Simulation, Curriculum


Citation: Roberts KE, Bell RL, Duffy AJ. Evolution of surgical skills training. World J Gastroenterol 2006; 12(20): 3219-3224
INTRODUCTION

“See one, do one, teach one” is one way to paraphrase what has been the traditional teaching method of surgical education in the operating rooms of the United States for over a hundred years. This longstanding dogma is being increasingly challenged by legal and ethical concerns for patient safety, 80-h work week training restrictions, the cost of operating room time, and surgical complications. The emerging field of surgical simulation and training offers an opportunity to teach and practice laparoscopic skills outside of the operating room environment. There is enormous potential to address patient safety, risk management concerns, OR management, and work hours requirements with more efficient and effective training methods. The current goal of simulator training is to help trainees acquire the skills needed to perform complex minimally invasive surgical procedures prior to practicing them on living patients. For abdominal surgeons, minimally invasive surgery primarily involves laparoscopic techniques. The concept of minimally invasive surgery also applies to thoracic, gynecologic, head and neck surgery, orthopedics, or any other field where the size of the incisions and the degree of injury to the patient can be minimized with advanced techniques and equipment.

Novel methods of developing these skill sets are necessary because it is becoming increasingly clear that minimally invasive surgery requires a completely different skill set from the one used in traditional, open surgery[1-3]. Here spatial relationships and associated psychomotor skills are necessary to manipulate surgical instruments on a two-dimensional video screen in an actual three-dimensional operative field[4,5]. Developing ambidextrous skills in the small intraabdominal space, handicapped by instruments that have limited degree of freedom[6], while compensating for difficult camera angles and the fulcrum effect[7] often seems to be an overwhelming task for novices in the field of laparoscopic surgery. Unfortunately, even with the restrictions on work hours, we are now teaching residents at least two ways of performing each procedure, essentially doubling the skills that have to be learned over a five year residency.

Over the last few decades laparoscopic procedures have evolved from diagnostic laparoscopy[8] to advanced, more complex procedures. It is now routine for minimally invasive techniques to be used for bariatric, colonic, and advanced foregut procedures. These more advanced procedures require highly developed psychomotor skills. The surgeon’s anatomic awareness must be developed in concert with the ability to safely achieve exposure and identify and control important structures. To assist with the critical components of many advanced operations, specialized equipment is available and commonly used. These instruments require a great deal of expertise in order to be used effectively and safely. Anecdotal reports have described significant complications related to the incorrect use of such equipment[9,10]. These developments have not gone unnoticed. The Food and Drug Administration (FDA) has now inserted itself into the training debate by requiring the development of a simulator-based training program (and that physicians demonstrate competence on the device) before approving a novel vascular stenting device. This ruling, in April of 2004, represents the first mandate by the FDA to require simulation training[11] without fully established or accepted data proving the validity of this approach. Similar requirements at the federal level for other advanced devices and skills sets are likely to follow. Payer-mandated and hospital-based credentialing for procedures and equipment is undoubtedly not far behind.

Surgeons who teach and train advanced procedures must now help develop methods of training and evaluation that truly establish procedural based competency. The tools are being developed and it is up to physician-educators to maximize their potential and set the standards for excellence. In a healthcare environment focused on evidence-based medicine and outcomes data, surgeons should set the standards rather than have them imposed by outside interests. The bar has already been set; the Accreditation Council for Graduate Medical Education (ACGME) now requires that program directors provide a statement attesting to the competency of every graduating Chief Surgical resident[12]. Many tools are now available to help with this task. The technology exists to continue to improve the realism and scope of training devices. Standardizing training routines and the effectively integrating systems for evaluation and credentialing is becoming a realistic goal. Below we will review currently available training and evaluation devices for laparoscopic surgery and discuss current future implications for training curricula for surgical residents.

BOX TRAINERS

This type of surgical simulator uses real surgical instruments and equipment including video monitors, cameras, and laparoscopes (Figure 1). It is an opaque box that approximates the size of the adult human abdominal cavity. Slits are prefabricated on the anterior surface of the box, through which trocars (access ports) may be placed. An attached flexible arm acts as a camera holder. Laparoscopic instruments are then inserted through the ports and into the box. Various targets are manipulated inside the box, with visual information relayed through a video source and display comparable to that used in most operating theatres. Tactile feedback is limited, as it is in laparoscopic surgery, by the instruments used. The use of real instruments and equipment is clearly the strength of these systems. However, the drills developed lack the face validity offered by other systems; the instruments may be real, but the “tissues” used clearly are not.

Figure 1
Figure 1 Typical Box trainer with real surgical instruments and equipment including a video monitor, camera, and laparoscope.

Many drills have been developed for use on these trainers. These drills traditionally involve movement and coordination exercises using peg boards or small objects such as beans or sugar cubes. Suturing skills are practiced using standard sutures and needles through plastic drains or other simulated tissue. Cadaveric tissue specimens can be placed inside the box to improve the realism of some tasks, particularly suturing. Practicing skills and tasks in the box trainer that resemble parts of real operations has been shown to stimulate the learning of psychomotor skills[13]. The physical sensory feedback conferred via the instruments in a box trainer is equivalent to that of surgery. The feel of the instruments on the surfaces of tissue, the pressure on closing a handle, and the compliance of compressed structures evokes feedback sensations called haptics[14]. This sensory feedback is one of the important attributes of the box trainer simulator.

Low acquisition cost is another key attribute of these devices. This makes the box trainer the most widely available training system. Its cost and availability have helped establish this type of simulator as one of the most validated systems in surgical training and evaluation. The use of these trainers has been well documented as part of the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS)[15]. This system has been extensive studied and validated and has evolved into the Society of Gastrointestinal and Endoscopic Surgeons (SAGES) Fundamentals of Laparoscopic Surgery (FLS) course and evaluation system[1]. The FLS course combines an interactive CD-ROM educational tool that emphasizes basic principles of laparoscopic surgery with training drills for the box trainer.

The box trainer provides the backbone of the FLS evaluation tool. The FLS exam combines a written exam with a timed and scored laparoscopic skills evaluation in the box trainer. The tasks used for laparoscopic skill evaluation are peg transfer, pattern cutting, placement of a ligating loop, intracorporeal and extracorporeal knot tying[15,16]. SAGES is currently recommending that all senior residents demonstrate laparoscopic skill competency by taking this exam. An examination fee is charged and, currently, the exam is only offered at six North American testing centers. The exam can also be taken at the yearly American College of Surgeons (ACS) Clinical Congress and the SAGES annual meeting.

The need to take a box trainer-based exam at a specialized test center demonstrates the major disadvantage of this system. The measurement of performance and objective evaluation of skills, or metrics, requires an independent teacher/evaluator[17]. For FLS, objective evaluation is performed with trained test proctors who time performance and apply penalties to the performance based on standardized criteria[18,19]. These criteria have not been published or made widely available, limiting the use of this validated evaluation system to only the approved test centers. As such, the FLS system is not useful as a routine evaluation tool for individual residency programs.

ANIMAL MODELS

These simulators involve the use of a live, anesthetized animal. This is the most realistic, non-patient environment for laparoscopic training. The abdomen in the porcine model is comparable in size to the adult human, with much of the foregut anatomy similar to that of the human[20]. Performing a cholecystectomy in this model provides tactile feedback in an environment where technical errors and complications such as gallbladder perforation or common bile duct injury can occur without consequence to a human patient[21]. Likewise, the canine model is frequently used to practice colon surgery. Animal models also enable trainees to work together as a team on an operation, providing additional insight into setting up an operative case.

As useful as animal labs are, there are many reasons why they are not fully integrated into most surgical curricula. Ethical issues regarding the use of animals for training and studies are not to be discounted, but for most programs the cost issues are prohibitive. There are substantial costs associated with maintaining specialized facilities and providing appropriate staff[22]. In some institutions, including our own, these facilities have been converted into an inanimate skills training lab.

VIRTUAL REALITY SIMULATORS

Virtual reality (VR) surgical simulators are the latest and most promising development in the area of surgical simulation. Sophisticated computer software has been developed in an attempt to replicate critical skills required for laparoscopic surgery and, in some cases, entire procedures. Many of these simulators provide a more believable practice environment than traditional box trainers, hence providing higher face validity. These trainers can be set up to record and save accurate and objective data for individual performance on specific tasks. The metrics of most devices can be customized, setting pass/fail criteria. These features present the opportunity for a trainee to practice independently on their own time as part of a structured curriculum.

The performance records make it possible for the educator to evaluate the performance of a laparoscopic task in an easy accessible format, to track the progress of an individual, and to compare a trainees results to peers and an expert standard[6,23,24]. Virtual reality simulators are also available for technically challenging tasks such as upper and lower endoscopy, endoscopic retrograde cholangiopancreatography (ERCP), hysteroscopy, and cystoscopy[24-28]. Objective measurements such as the time to complete a task, economy of hand motion, dexterity, and instrument path length can be easily used as assessment tools to document the progress of laparoscopic skills. The software of new VR simulators can be adjusted to create more difficult tasks or different levels of difficulty of the same task. Software updates have the potential to provide new drills or procedures which can be simulated using the same hardware. In an attempt to replicate the biggest advantage of box trainers and make the simulations as real as possible, several VR simulators now offer built in haptics, or force feedback, as an option on their systems. These systems, while intriguing, have not yet been shown to significantly contribute to training[29] but do significantly contribute to the cost of the devices.

Currently the following VR simulators are commercially available as training platforms for laparoscopic surgery: Procedicus MIST[30], Haptica ProMIS[31], METI SurgicalSIM, Simbionix Lap-Mentor and Surgical Science LapSim[32]. The two most widely used VR simulators are described in more detail below. The Procedicus MIST (Minimally Invasive Surgical Trainer), the most recent version of the MIST-VR (Mentice AB, Gothenburg, Sweden), uses a desktop computer with an interface similar to two laparoscopic instruments[7]. Twelve different tasks designed to improve hand-eye coordination and to stimulate psychomotor skills constitute the basis of this system. The complexity of these tasks ranges from basic grasping targets to transferring them between instruments to performing these tasks with an increased difficulty level, including the use of diathermy[33]. Recent updates to the system have incorporated suturing and knot tying drills. The MIST system remains the most extensively validated virtual reality training system. Seymour et al[34] demonstrated that surgical trainees who practiced a diathermy task to an expected level of performance performed faster and with fewer errors in excising the gallbladder off the liver.

The LapSim system (Surgical Science, Göteborg, Sweden) (Figure 2) also consists of laparoscopic instruments (VLI interface, Immersion Medical, Gaithersburg, MD) connected to a desktop computer (Figure 2). Similar to the MIST, the LapSim focuses on basic laparoscopic skills and suturing. Newer software modules incorporate complex diathermy and bowel manipulation. The LapSim system has different modules mimicking different tasks in the abdominal cavity focusing on camera navigation, instrument navigation, coordination, grasping, cutting, clip applying and more advanced modules like suturing and “running the small bowel.” The simulator records data on multiple parameters for each module including “pass/fail,” tissue damage and excess instrument motion[35,36]. All of the various data sets are easily accessible and downloadable with the latest software release. This same software allows a teacher to set up specific drills and tasks as part of a curriculum for each trainee. All the metrics of this system are customizable. Likewise, performance exams can be tailored to skill level or individual requirements. Performance on the LapSim system has been demonstrated to distinguish expert and novice skill levels. This establishes construct validity for this system[36]. Specific skills transfer to the operating room has not yet been demonstrated.

Figure 2
Figure 2 The LapSim virtual reality system: The laparoscopic VLI interface with a PC-based desktop computer.
FULL PROCEDURAL SIMULATORS

Full procedural simulators go beyond the basic laparoscopic training of their predecessors and are designed to recreate the specific anatomy allowing the practicing of all skills necessary to perform a particular operation. There are a variety of simulators available for endoscopic sinus surgery, bronchoscopy, endovascular surgery, gastrointestinal endoscopy, and laparoscopic procedures such as cholecystectomy and inguinal hernia repair[37-40]. Many of these simulators are part of a computer-based virtual reality platform. The hernia repair simulator is a physical model simulator[39].

The potential advantage of full procedure simulators over basic skills simulators is obvious. A well designed full procedural simulation should teach skills, anatomy, and the nuances of a complete surgical procedure in an environment where errors can be made without consequence. A full procedure simulator will not replace basic skills drills but will help bridge the gap to the operating room. For instance, in a laparoscopic total extraperitoneal hernia repair (TEP) even an experienced surgeon might encounter difficulties during repair due to the different approach and the unusual view of the inguinal anatomy. A full procedural TEP-simulator in the training of surgical residents resulted not only in improvement of operative performance but with their new understanding the residents were also more willing to recommend this procedure to patients requiring an inguinal hernia repair[39].

TRAINING OF SURGICAL RESIDENTS

In 1999 the Accreditation Council of Graduate Medical Education (ACGME) endorsed the concept of six core competences which every physician should demonstrate: Patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice[41]. These competencies are now the basis of individualized evaluations in most residency training programs. These competencies do not directly address the acquisition of the extensive technical skills required for qualified surgeons or other practitioners of technical specialties. However, these skills are included across most of the competencies and one goal of the ACGME initiative is for training programs to develop novel and accurate measures of these competencies in practice. In 2001, a survey confirmed that the vast majority of program directors in general surgery are in favor of surgical training outside the operating room to augment and complement hands-on experience with real patients[42]. The discrepancy between the demand for guidelines for laparoscopic simulator training for surgical residents and the application of these methods is telling. Curriculum development and validation of these methods is now of the utmost priority.

The first nationwide curriculum aiming to address this need is described in the FLS program[1]. This program is available for purchase by individual programs but is primarily designed as an independent evaluation tool. The FLS curriculum does not provide a mechanism for purchasing programs to internally evaluate resident performance or recommend distinct training guidelines. In addition, the FLS was deliberately designed to not include virtual reality computer simulators because of the acquisition costs involved and the lack of standardization and complete validation studies[1]. SAGES intends the FLS to become the standard for evaluation of basic laparoscopic skill acquisition but FLS may have limited long-term appeal if it does not incorporate virtual reality technology as it evolves.

To justify the steep acquisition costs, the validity and reliability of new teaching and evaluation tools must be supported by convincing studies before their widespread acceptance can be expected. Up until now, most skill evaluation for laparoscopic training and, in fact, surgery training in general, has been based on subjective evaluation and recommendation, a system that has been criticized for its lack of validity[43]. More recently simulation systems are being used in some training programs for the objective assessment of individual technical performance and as a supplement to the traditional evaluation[18,44-46]. An important aspect of a valid training system is that novices show an improvement of their skills over time. This has been demonstrated in multiple studies to be correct for the MISTELS, and more recently, for the VR-simulators[32,34,36,47]. Interestingly, some studies seem to support the work hours mandates. VR simulators have been used to demonstrate a decrease in psychomotor skills in residents subjected to sleep deprivation from a night of trauma call. Although the time to complete the tasks was unchanged, the number of errors was increased[48]. Another interesting study shows that alcohol intoxicated medical students perform worse in a VR simulator and regain their capabilities once sober[49]. The conclusions of these studies are quite serious and they rely on the discriminative validity of the simulator used to assess differing performance.

Despite strong evidence supporting the use of surgical simulators in minimally invasive training of surgical residents, the learning curves to acquire psychomotor skills are relatively unknown within the field of surgical simulation. It is reasonable to think that “distributed practice” or frequent, repeated training provides longer term skill acquisition versus “massed practice”, or intense short term training[50]. In a study evaluating the learning curve in laparoscopic skill training, different learning curves existed for surgeons with different laparoscopic backgrounds[24]. Much remains to be learned in determining the most efficient methods of expanding psychomotor skills. Contributing factors need to be identified and evaluated to determine the specific types of exercises that lead to improved performance in the operating room. The optimum practice time and requirements for continued practice need to be delineated.

MAINTENANCE OF SKILLS AND CREDENTIALING OF SURGEONS

As technologies and procedures evolve, newer techniques may require additional specialized skills. How do hospitals, practices, other physicians, and patients ensure that surgeons possess the requisite skills to perform these procedures safely and deal with intraoperative complications How do we ensure the surgeons retain the skills to maintain a high level of practice These questions surfaced before the advent of laparoscopic abdominal surgery and persist today with greater urgency because of the requirements and demands of modern practice[51]. The American Board of Surgery (ABS) is considering an objective skills proficiency examination to supplement the current knowledge-based and judgment assessments that comprise the surgery Qualifying and Certification exams. A completely objective, reproducible, and universally accepted evaluation tool is currently lacking, but it is likely that such a system will be developed and implemented in our practice lifetime. Similar to the aviation industry, a surgeon’s skills at handling a difficult or emergent problem could be assessed. An exam could incorporate assessment of judgment and anatomic awareness. This exam could be employed for ongoing assessment of skill level at predefined intervals, much like the current recertification exam system. Patients entrust their lives to surgeons who rely on their judgment, visual and psychomotor skills. In this aspect, surgeons share common ground with airline pilots and it seems logical that surgeons, too, should be involved in an established method of acquiring and maintaining their high level technical skills. Surgical simulators will likely become an indispensable tool to these ends. The standards and performance expectations for such a system should be designed and validated by surgeons and surgeon-educators to ensure the exams are fair and reproducible.

The surgery of the future

Surgical simulators have the potential to be much more than tools for training and evaluation. As the technology continues to develop, the high standards required for appraisal and certification should allow future generations of simulators to also be used for operative planning. Complex operations will be planned and practiced in a virtual reality environment based on three dimensional imaging technologies imported into the device. This will be enabled by the same technology that allows us instant access to high-resolution computed tomography (CT) and magnetic resonance imaging (MRI) at our desktops today. After a surgeon has sufficiently practiced the virtual procedure, he would then perform the actual operation in the patient with significantly less operative morbidity, or perhaps with augmented-reality computer guidance[52]. The potential benefits to the patient are enormous.

In conclusion, surgical skills training is undergoing a dramatic transformation. Modern technology and training techniques, combined with external pressures and mandates are forcing surgeon-educators to rethink previously held principles. Simulators in their current form can facilitate the improvement of laparoscopic skills and operating room performance of surgical residents. Development of standardized training curricula for surgical residents remains an urgent and important agenda, particularly for minimal invasive surgery. An innovative and progressive approach, learning from the experiences of the field of aviation, can provide the foundation for the next century of surgical training. As the technology develops, the way we practice will continue to evolve, to the benefit of physicians and patients.

Footnotes

S- Editor Pan BR E- Editor Bi L

References
1.  Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hoffman K. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery. 2004;135:21-27.  [PubMed]  [DOI]
2.  Friedman RL, Pace BW. Resident education in laparoscopic cholecystectomy. Surg Endosc. 1996;10:26-28.  [PubMed]  [DOI]
3.  Figert PL, Park AE, Witzke DB, Schwartz RW. Transfer of training in acquiring laparoscopic skills. J Am Coll Surg. 2001;193:533-537.  [PubMed]  [DOI]
4.  Gallagher AG, Cowie R, Crothers I, Jordan-Black JA, Satava RM. PicSOr: an objective test of perceptual skill that predicts laparoscopic technical skill in three initial studies of laparoscopic performance. Surg Endosc. 2003;17:1468-1471.  [PubMed]  [DOI]
5.  Wanzel KR, Hamstra SJ, Caminiti MF, Anastakis DJ, Grober ED, Reznick RK. Visual-spatial ability correlates with efficiency of hand motion and successful surgical performance. Surgery. 2003;134:750-757.  [PubMed]  [DOI]
6.  Munz Y, Kumar BD, Moorthy K, Bann S, Darzi A. Laparoscopic virtual reality and box trainers: is one superior to the other. Surg Endosc. 2004;18:485-494.  [PubMed]  [DOI]
7.  Gallagher AG, McClure N, McGuigan J, Crothers I, Browning J. Virtual reality training in laparoscopic surgery: a preliminary assessment of minimally invasive surgical trainer virtual reality (MIST VR). Endoscopy. 1999;31:310-313.  [PubMed]  [DOI]
8.  Kelling G. Ueber oesophagoskopie, gastroskopie und kolioskopie. Munch Med Wochenschr. 1902;1:21-24.  [PubMed]  [DOI]
9.  Szwerc MF, Landreneau RJ, Santos RS, Keenan RJ, Murray GF. Minithoracotomy combined with mechanically stapled bronchial and vascular ligation for anatomical lung resection. Ann Thorac Surg. 2004;77:1904-1909; discussion 1909-1910.  [PubMed]  [DOI]
10.  Deng DY, Meng MV, Nguyen HT, Bellman GC, Stoller ML. Laparoscopic linear cutting stapler failure. Urology. 2002;60:415-419; discussion 419-420;.  [PubMed]  [DOI]
11.  Gallagher AG, Cates CU. Approval of virtual reality training for carotid stenting: what this means for procedural-based medicine. JAMA. 2004;292:3024-3026.  [PubMed]  [DOI]
12.  ACGME .   Common Program Requiremnets. 2005. Available from: http://www.acgme.org/acWebsite/navPages/nav_PDcoord.asp.  [PubMed]  [DOI]
13.  Anastakis DJ, Regehr G, Reznick RK, Cusimano M, Murnaghan J, Brown M, Hutchison C. Assessment of technical skills transfer from the bench training model to the human model. Am J Surg. 1999;177:167-170.  [PubMed]  [DOI]
14.  Bholat OS, Haluck RS, Murray WB, Gorman PJ, Krummel TM. Tactile feedback is present during minimally invasive surgery. J Am Coll Surg. 1999;189:349-355.  [PubMed]  [DOI]
15.  Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG. Proving the value of simulation in laparoscopic surgery. Ann Surg. 2004;240:518-525; discussion 525-528.  [PubMed]  [DOI]
16.  Derossis AM, Fried GM, Abrahamowicz M, Sigman HH, Barkun JS, Meakins JL. Development of a model for training and evaluation of laparoscopic skills. Am J Surg. 1998;175:482-487.  [PubMed]  [DOI]
17.  Madan AK, Frantzides CT, Shervin N, Tebbit CL. Assessment of individual hand performance in box trainers compared to virtual reality trainers. Am Surg. 2003;69:1112-1114.  [PubMed]  [DOI]
18.  Fraser SA, Klassen DR, Feldman LS, Ghitulescu GA, Stanbridge D, Fried GM. Evaluating laparoscopic skills: setting the pass/fail score for the MISTELS system. Surg Endosc. 2003;17:964-967.  [PubMed]  [DOI]
19.  Keyser EJ, Derossis AM, Antoniuk M, Sigman HH, Fried GM. A simplified simulator for the training and evaluation of laparoscopic skills. Surg Endosc. 2000;14:149-153.  [PubMed]  [DOI]
20.  Watson DI, Treacy PJ, Williams JA. Developing a training model for laparoscopic common bile duct surgery. Surg Endosc. 1995;9:1116-1118.  [PubMed]  [DOI]
21.  Wolfe BM, Szabo Z, Moran ME, Chan P, Hunter JG. Training for minimally invasive surgery. Need for surgical skills. Surg Endosc. 1993;7:93-95.  [PubMed]  [DOI]
22.  Gruber FP, Hartung T. Alternatives to animal experimentation in basic research. ALTEX. 2004;21 Suppl 1:3-31.  [PubMed]  [DOI]
23.  Brunner WC, Korndorffer JR Jr, Sierra R, Massarweh NN, Dunne JB, Yau CL, Scott DJ. Laparoscopic virtual reality training: are 30 repetitions enough. J Surg Res. 2004;122:150-156.  [PubMed]  [DOI]
24.  Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J. Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills. Am J Surg. 2003;185:146-149.  [PubMed]  [DOI]
25.  Ritter EM, McClusky DA 3rd, Lederman AB, Gallagher AG, Smith CD. Objective psychomotor skills assessment of experienced and novice flexible endoscopists with a virtual reality simulator. J Gastrointest Surg. 2003;7:871-877; discussion 877-878;.  [PubMed]  [DOI]
26.  Voss G, Bockholt U, Los Arcos JL, Müller W, Oppelt P, Stähler J. LAHYSTOTRAIN intelligent training system for laparoscopy and hysteroscopy. Stud Health Technol Inform. 2000;70:359-364.  [PubMed]  [DOI]
27.  Shah J, Darzi A. Virtual reality flexible cystoscopy: a validation study. BJU Int. 2002;90:828-832.  [PubMed]  [DOI]
28.  Sedlack R, Petersen B, Binmoeller K, Kolars J. A direct comparison of ERCP teaching models. Gastrointest Endosc. 2003;57:886-890.  [PubMed]  [DOI]
29.  Maass H, Chantier BB, Cakmak HK, Trantakis C, Kuehnapfel UG. Fundamentals of force feedback and application to a surgery simulator. Comput Aided Surg. 2003;8:283-291.  [PubMed]  [DOI]
30.  Ström P, Kjellin A, Hedman L, Johnson E, Wredmark T, Felländer-Tsai L. Validation and learning in the Procedicus KSA virtual reality surgical simulator. Surg Endosc. 2003;17:227-231.  [PubMed]  [DOI]
31.  Van Sickle KR, McClusky DA 3rd, Gallagher AG, Smith CD. Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surg Endosc. 2005;19:1227-1231.  [PubMed]  [DOI]
32.  Duffy AJ, Hogle NJ, McCarthy H, Lew JI, Egan A, Christos P, Fowler DL. Construct validity for the LAPSIM laparoscopic surgical simulator. Surg Endosc. 2005;19:401-405.  [PubMed]  [DOI]
33.  Langelotz C, Kilian M, Paul C, Schwenk W. LapSim virtual reality laparoscopic simulator reflects clinical experience in German surgeons. Langenbecks Arch Surg. 2005;390:534-537.  [PubMed]  [DOI]
34.  Chaudhry A, Sutton C, Wood J, Stone R, McCloy R. Learning rate for laparoscopic surgical skills on MIST VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg Engl. 1999;81:281-286.  [PubMed]  [DOI]
35.  Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, Satava RM. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002;236:458-463; discussion 463-464.  [PubMed]  [DOI]
36.  Hyltander A, Liljegren E, Rhodin PH, Lönroth H. The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc. 2002;16:1324-1328.  [PubMed]  [DOI]
37.  Edmond CV Jr. Impact of the endoscopic sinus surgical simulator on operating room performance. Laryngoscope. 2002;112:1148-1158.  [PubMed]  [DOI]
38.  Schaefer JJ 3rd. Simulators and difficult airway management skills. Paediatr Anaesth. 2004;14:28-37.  [PubMed]  [DOI]
39.  Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV, Tesfay ST, Jones DB. Improving operative performance using a laparoscopic hernia simulator. Am J Surg. 2001;182:725-728.  [PubMed]  [DOI]
40.  Hsu JH, Younan D, Pandalai S, Gillespie BT, Jain RA, Schippert DW, Narins CR, Khanna A, Surowiec SM, Davies MG. Use of computer simulation for determining endovascular skill levels in a carotid stenting model. J Vasc Surg. 2004;40:1118-1125.  [PubMed]  [DOI]
41.  ACGME .   Accreditation Council for Graduate Medical Education (ACGME) General Competencies. Version 1.3. Chicago, IL: 2000.  [PubMed]  [DOI]
42.  Haluck RS, Marshall RL, Krummel TM, Melkonian MG. Are surgery training programs ready for virtual reality A survey of program directors in general surgery. J Am Coll Surg. 2001;193:660-665.  [PubMed]  [DOI]
43.  Wanzel KR, Ward M, Reznick RK. Teaching the surgical craft: From selection to certification. Curr Probl Surg. 2002;39:573-659.  [PubMed]  [DOI]
44.  Derossis AM, Antoniuk M, Fried GM. Evaluation of laparoscopic skills: a 2-year follow-up during residency training. Can J Surg. 1999;42:293-296.  [PubMed]  [DOI]
45.  Feldman LS, Hagarty SE, Ghitulescu G, Stanbridge D, Fried GM. Relationship between objective assessment of technical skills and subjective in-training evaluations in surgical residents. J Am Coll Surg. 2004;198:105-110.  [PubMed]  [DOI]
46.  Sidhu RS, Grober ED, Musselman LJ, Reznick RK. Assessing competency in surgery: where to begin. Surgery. 2004;135:6-20.  [PubMed]  [DOI]
47.  Sherman V, Feldman LS, Stanbridge D, Kazmi R, Fried GM. Assessing the learning curve for the acquisition of laparoscopic skills on a virtual reality simulator. Surg Endosc. 2005;19:678-682.  [PubMed]  [DOI]
48.  Eastridge BJ, Hamilton EC, O'Keefe GE, Rege RV, Valentine RJ, Jones DJ, Tesfay S, Thal ER. Effect of sleep deprivation on the performance of simulated laparoscopic surgical skill. Am J Surg. 2003;186:169-174.  [PubMed]  [DOI]
49.  Dorafshar AH, O'Boyle DJ, McCloy RF. Effects of a moderate dose of alcohol on simulated laparoscopic surgical performance. Surg Endosc. 2002;16:1753-1758.  [PubMed]  [DOI]
50.  Mackay S, Morgan P, Datta V, Chang A, Darzi A. Practice distribution in procedural skills training: a randomized controlled trial. Surg Endosc. 2002;16:957-961.  [PubMed]  [DOI]
51.  Dent TL. Training, credentialling, and granting of clinical privileges for laparoscopic general surgery. Am J Surg. 1991;161:399-403.  [PubMed]  [DOI]
52.  Maresceaux J, Soler L, Ceulemans R, Garcia A, Henri M, Dutson E. [Image fusion, virtual reality, robotics and navigation. Effects on surgical practice]. Chirurg. 2002;73:422-427.  [PubMed]  [DOI]