Minireviews Open Access
Copyright ©The Author(s) 2018. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastroenterol. Nov 28, 2018; 24(44): 4974-4978
Published online Nov 28, 2018. doi: 10.3748/wjg.v24.i44.4974
Learning curves in minimally invasive esophagectomy
Frans van Workum, Camiel Rosman, Department of Surgery, Radboud University Medical Center, Nijmegen 6500 HB, Netherlands
Laura Fransen, Misha DP Luyer, Department of Surgery, Catharina Hospital, Eindhoven 5602 ZA, Netherlands
ORCID number: Frans van Workum (0000-0003-0623-9066); Laura Fransen (0000-0002-8404-673X); Misha DP Luyer (0000-0002-9483-1520); Camiel Rosman (0000-0002-9152-3987).
Author contributions: van Workum F, Luyer MDP and Rosman C designed research; van Workum F and Fransen L performed research; van Workum F analyzed data; all authors wrote the paper.
Conflict-of-interest statement: The authors have no conflict of interest to declare. The work has not been previously published and has not been submitted for publication elsewhere.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:
Corresponding author to: Frans van Workum, MD, Academic Research, Doctor, Department of Surgery, Radboud University Medical Center, Geert Grooteplein-Zuid 10, Nijmegen 6500 HB, Netherlands.
Telephone: +31-24-3611111 Fax: +31-24-3540501
Received: August 25, 2018
Peer-review started: August 27, 2018
First decision: October 11, 2018
Revised: October 28, 2018
Accepted: November 2, 2018
Article in press: November 2, 2018
Published online: November 28, 2018


Surgical innovation and pioneering are important for improving patient outcome, but can be associated with learning curves. Although learning curves in surgery are a recognized problem, the impact of surgical learning curves is increasing, due to increasing complexity of innovative surgical procedures, the rapid rate at which new interventions are implemented and a decrease in relative effectiveness of new interventions compared to old interventions. For minimally invasive esophagectomy (MIE), there is now robust evidence that implementation can lead to significant learning associated morbidity (morbidity during a learning curve, that could have been avoided if patients were operated by surgeons that have completed the learning curve). This article provides an overview of the evidence of the impact of learning curves after implementation of MIE. In addition, caveats for implementation and available evidence regarding factors that are important for safe implementation and safe pioneering of MIE are discussed.

Key Words: Minimally invasive esophagectomy, Learning curve, Pioneering, Safe implementation, Proficiency gain curve

Core tip: Surgical innovation and pioneering are important for improving patient outcome, but can be associated with learning curves. The impact of surgical learning curves is increasing, due to increasing complexity of innovative surgical procedures and the rapid rate at which new interventions are implemented. Learning curves of minimally invasive esophagectomy can take years to complete and evidence based training and safe implementation programs are paramount to decrease implementation associated morbidity.


A surgical learning curve is a phase after implementation of a new procedure, that is characterized by improving performance as experience with the new procedure increases[1,2]. Learning curves were first described in aviation and manufacturing science[3]. but it has become a widely used concept in medicine and surgery[1,4,5]. Although surgical innovation is necessary to improve care, it is important to take surgical learning curves into account since they have been associated with a negative impact on patient outcome[6-8].

An emerging problem of ongoing surgical innovation

Surgeons are aware of the existence of learning curves, the beneficial effects of “learning before doing” and the importance of safely implementing new surgical procedures. However, ongoing surgical innovation is presenting new challenges regarding surgical learning curves, since new interventions are associated with increasing surgical complexity and decreasing relative effectiveness compared to older procedures.

For example, when the tension-free mesh repair was introduced for inguinal hernias, this lead to a dramatically lower incidence of hernia recurrence compared to conventional non-mesh repairs[9,10]. The large difference in relative effectiveness and the limited complexity (associated with short learning curves) of tension-free mesh repair surgery contributed to making the learning curve insignificant for this procedure (Figure 1A). However, surgical procedures that are currently implemented are of a higher complexity[4,11], are associated with longer learning curves, and the new procedures have a much lower relative effectiveness benefit compared to the old procedures (Figure 1B). For example, trials comparing laparoscopic vs open gastrointestinal procedures have shown more modest improvements in outcome for patients[12-14] and the difference in relative effectiveness is even smaller in trials comparing robotic vs laparoscopic procedures[15].

Figure 1
Figure 1 Scenarios in which the differences in the impact that learning curves can have on the effectiveness of an innovative intervention is described. A: Learning curves can be neglected in case of a short learning curve and large difference in relative effectiveness between the regular technique and the innovative technique. B: If the learning curve of an innovative technique is substantial and the difference in relative effectiveness is small, learning curves can have a large impact on when an innovative technique becomes effective.

These developments have contributed to the situation in which the clinical effectiveness of a new surgical procedure has become more dependent on the delivery of the treatment. In addition, new interventions are implemented at an increasing rate, driven by the patient’s increasing expectation to be operated by the newest, most technically challenging, minimally invasive procedures[11]. Together, these factors have contributed to the situation in which learning curves have become more important in contemporary surgical practice. The impact of learning curves is likely to become even more significant for patient outcome in the near future, as surgical innovation progresses further. Although surgeons are always searching for the best way to treat their patients and innovation is needed to further improve surgical care, implementation of technically challenging procedures may come at a price.


For esophagectomy, beneficial effects of minimally invasive surgery have been well documented[13,14,16]. However, extensive learning curve effects of minimally invasive esophagectomy (MIE) have also been described. Earlier MIE learning curve studies have focused on outcome parameters directly related to the procedure itself, such as blood loss and operative time[17-19]. More recently however, the effect of learning curves on clinically relevant outcome measures has been established for anastomotic leakage[8], mortality[6], and survival[7]. Learning associated morbidity (morbidity during a learning curve, that could have been avoided if patients were operated by surgeons that have completed the learning curve)[8] is now a recognized problem, since there is accumulating data that learning curves have significant impact on critical outcome parameters. Despite beneficial results of MIE compared to older techniques, this implicates that there is significant room for improvement regarding patient safety during surgical learning curves. However, it can take years to become proficient in MIE with low postoperative morbidity and possibly the impact of learning associated morbidity is greater than the direct benefit of MIE compared to open esophagectomy during the learning curve phase[6,8,13,14]. This puts the effectiveness of recent innovations into perspective and exposes the importance of ensuring safety during learning curves. In addition, the evidence of the impact of learning curves and learning associated morbidity comes with the opportunity and obligation to determine what factors contribute to safer implementation and to investigate how to shorten the learning curve and decrease learning associated morbidity. It might be hypothesized that this type of research is more beneficial to patients than research into new innovations in the current time.

Another important consideration is that it is plausible that various types of MIE with different levels of complexity (i.e., transhiatal, transthoracic with cervical anastomosis and transthoracic with intrathoracic anastomosis), have different learning curves. Although these differences have not been exposed clearly in studies, it is likely that they result in differences in learning associated morbidity. This may also be true for hybrid MIE, in which either the thoracic or abdominal phase is performed by open surgery. For example, by performing an open intrathoracic anastomosis instead of a thoracoscopic anastomosis, a surgeon can omit performing the most important and complex part of the procedure in a technically more demanding, thoracoscopic fashion. This may shorten the learning curve and reduce learning associated morbidity, but no studies have been published that support this hypothesis. Together with data from studies comparing the effectiveness of different approaches of MIE, data on differences in learning curves can be used by clinicians to decide which approach is best to implement in their practice.


Implementation of increasingly complex innovative procedures require increasingly effective and safe implementation programs to prevent learning associated morbidity. Standardized training programs have been described to be effective for surgical procedures[20-22]. Although the implementation of a new, technically challenging surgical procedure probably requires a multidisciplinary and multifaceted approach, there is currently very little robust evidence on how an implementation program should be designed and what factors make them effective[23]. In addition, the effectiveness of interventions aimed at safer implementation has not been adequately compared to implementation without these interventions. For example, proctorship is widely used to shorten the learning curve and ensure safe implementation of a new technique, but to the best of our knowledge, it’s effectiveness has not been compared to implementation without a proctor.

Research that focuses on identification of factors that are associated with shorter learning curves and lower learning associated morbidity is important. Recently, we investigated whether surgeon age and surgeon volume were associated with the length of learning curves in patients undergoing open esophagectomy using a Swedish national esophagectomy registry[24]. In this study, younger surgeons and higher volume surgeons had shorter learning curves compared to older surgeons and lower volume surgeons. Although this study has its limitations, this is the first evidence that suggests that selecting surgeons and training them in high volume facilities may be beneficial to patients and enable safer implementation.

Safe implementation of MIE

For MIE, fundamental items of a safe implementation program have been established by expert opinion[25]. However, although all fundamental requirements of safe implementation were met, our group of 4 European expert centers found a significant learning curve effect after implementation of MIE: anastomotic leakage decreased from 28.9% to well below 5%[8]. Thirty-six patients (10.1% of all patients operated during the learning curve that took a mean of 119 cases to complete) experienced learning associated anastomotic leakage. The fact that significant learning associated anastomotic leakage occurred underlines the need for more research regarding safe implementation.

However, these data should be interpreted with care. Although learning associated morbidity was high in this study, centers switched anastomotic techniques from a cervical anastomosis (McKeown) to an intrathoracic anastomosis (Ivor Lewis) and the incidence of anastomotic leakage did not change initially, since it was already higher in patients with a cervical anastomosis. Therefore, it did not seem unethical to proceed with implementation of Ivor Lewis TMIE in these centers at that time. These data also show that innovation can ultimately be associated with a favorable outcome, since the national incidence of intrathoracic anastomotic leakage was around 17% in the Netherlands at the time the study was performed[26]. In our opinion, some learning associated morbidity is inevitable and in general, this can be justified if morbidity during learning curves does not exceed the morbidity associated with the old procedure. However, shortening learning curves and reducing learning associated morbidity remain important goals that can be beneficial to a significant number of patients.


For surgeons, pioneering with new procedures, proper training of the surgical team prior to implementation of a new procedure may not be possible. Pioneering surgeons should describe the development and refinement of new procedures according to the Idea, Development, Exploration, Assessment, Long-term monitoring (IDEAL) framework[27]. According to the IDEAL framework, surgical innovation starts with the Idea (IDEAL stage I) in which proof of concept and feasibility are investigated. In the Development (IDEAL stage IIa) and Exploration (IDEAL stage IIb) stages, the procedure is developed and refined. Innovations are compared to old techniques in the Assessment (IDEAL stage 3) and Long-term study (IDEAL stage IV) phases regarding short- and long-term outcome. IDEAL stage III and IV are further characterized by a stable surgical technique and the development of standardized training programs that can be followed by other surgeons. It is plausible that surgeons that start pioneering new techniques (i.e. implement innovations in IDEAL stage IIa or IIb) have longer learning curves that are associated with more learning associated morbidity than surgeons that implement innovations after the surgical technique has been refined.

However, it is currently not uncommon for contemporary surgeons to implement surgical procedures in IDEAL stage IIa or IIb. The patient’s desire to be operated by the newest procedures and the surgeon’s desire to offer the newest surgical innovations may contribute to a surgeon’s decision to pioneer with procedures of which it has not yet been determined how to optimally perform key steps or to implement procedures of which the relative effectiveness has not been established adequately and no standardized training programs exist.

Although pioneering surgeons are the cornerstone of surgical progress and their innovations have led to beneficial outcomes for numerous patients, it has become apparent that pioneering with technically challenging procedures can be hazardous. Although currently no guidelines exist, pioneering should probably be reserved for the absolute experts in the field with extensive experience of similar procedures. A solid outcome registration and regular multidisciplinary outcome meetings in which the results and refinements of the new procedure are discussed may contribute to safer pioneering, but consensus is lacking.


Although surgical innovation is important in improving patient outcome, the problem of learning curves and learning associated morbidity is increasing with ongoing surgical innovation and increasing complexity of newly implemented procedures. More insight in amendable factors determining the efficiency of surgical learning and safe implementation programs may increase patient safety and lead to better outcomes in the current surgical era.


Manuscript source: Invited manuscript

Specialty type: Gastroenterology and hepatology

Country of origin: Netherlands

Peer-review report classification

Grade A (Excellent): A

Grade B (Very good): 0

Grade C (Good): C

Grade D (Fair): 0

Grade E (Poor): 0

P- Reviewer: Wang Y, Watson DI S- Editor: Ma RY L- Editor: A E- Editor: Yin SY

1.  Hopper AN, Jamison MH, Lewis WG. Learning curves in surgical practice. Postgrad Med J. 2007;83:777-779.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 233]  [Cited by in F6Publishing: 281]  [Article Influence: 17.6]  [Reference Citation Analysis (0)]
2.  Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Monk AF, Russell IT. Assessment of the learning curve in health technologies. A systematic review. Int J Technol Assess Health Care. 2000;16:1095-1108.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 131]  [Cited by in F6Publishing: 139]  [Article Influence: 6.0]  [Reference Citation Analysis (0)]
3.  Wright TP. Factors affecting the cost of airplanes. J Aeronaut Sci. 1936;3:122–128.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1740]  [Cited by in F6Publishing: 1726]  [Article Influence: 19.6]  [Reference Citation Analysis (0)]
4.  Harrysson IJ, Cook J, Sirimanna P, Feldman LS, Darzi A, Aggarwal R. Systematic review of learning curves for minimally invasive abdominal surgery: a review of the methodology of data collection, depiction of outcomes, and statistical analysis. Ann Surg. 2014;260:37-45.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 80]  [Cited by in F6Publishing: 84]  [Article Influence: 8.4]  [Reference Citation Analysis (0)]
5.  Khan N, Abboudi H, Khan MS, Dasgupta P, Ahmed K. Measuring the surgical ‘learning curve’: methods, variables and competency. BJU Int. 2014;113:504-508.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 110]  [Cited by in F6Publishing: 136]  [Article Influence: 12.4]  [Reference Citation Analysis (0)]
6.  Mackenzie H, Markar SR, Askari A, Ni M, Faiz O, Hanna GB. National proficiency-gain curves for minimally invasive gastrointestinal cancer surgery. Br J Surg. 2016;103:88-96.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 41]  [Cited by in F6Publishing: 51]  [Article Influence: 5.7]  [Reference Citation Analysis (0)]
7.  Markar SR, Mackenzie H, Lagergren P, Hanna GB, Lagergren J. Surgical Proficiency Gain and Survival After Esophagectomy for Cancer. J Clin Oncol. 2016;34:1528-1536.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 61]  [Cited by in F6Publishing: 80]  [Article Influence: 10.0]  [Reference Citation Analysis (0)]
8.  van Workum F, Stenstra MHBC, Berkelmans GHK, Slaman AE, van Berge Henegouwen MI, Gisbertz SS, van den Wildenberg FJH, Polat F, Irino T, Nilsson M. Learning Curve and Associated Morbidity of Minimally Invasive Esophagectomy: A Retrospective Multicenter Study. Ann Surg. 2017; Epub ahead of print.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 131]  [Cited by in F6Publishing: 190]  [Article Influence: 38.0]  [Reference Citation Analysis (0)]
9.  Lichtenstein IL, Shulman AG, Amid PK, Montllor MM. The tension-free hernioplasty. Am J Surg. 1989;157:188-193.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 796]  [Cited by in F6Publishing: 706]  [Article Influence: 20.2]  [Reference Citation Analysis (0)]
10.  EU Hernia Trialists Collaboration. Mesh compared with non-mesh methods of open groin hernia repair: systematic review of randomized controlled trials. Br J Surg. 2000;87:854-859.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 243]  [Cited by in F6Publishing: 229]  [Article Influence: 9.5]  [Reference Citation Analysis (0)]
11.  Dankelman J. Increasing complexity of medical technology and consequences for training and for outcome of care. Accessed August 24, 2018 Av8il8ble from: URL: .  [PubMed]  [DOI]  [Cited in This Article: ]
12.  Veldkamp R, Kuhry E, Hop WC, Jeekel J, Kazemier G, Bonjer HJ, Haglind E, Påhlman L, Cuesta MA, Msika S, Morino M, Lacy AM; COlon cancer Laparoscopic or Open Resection Study Group (COLOR). Laparoscopic surgery versus open surgery for colon cancer: short-term outcomes of a randomised trial. Lancet Oncol. 2005;6:477-484.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1691]  [Cited by in F6Publishing: 1620]  [Article Influence: 85.3]  [Reference Citation Analysis (0)]
13.  Biere SS, van Berge Henegouwen MI, Maas KW, Bonavina L, Rosman C, Garcia JR, Gisbertz SS, Klinkenbijl JH, Hollmann MW, de Lange ES. Minimally invasive versus open oesophagectomy for patients with oesophageal cancer: a multicentre, open-label, randomised controlled trial. Lancet. 2012;379:1887-1892.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1067]  [Cited by in F6Publishing: 1127]  [Article Influence: 93.9]  [Reference Citation Analysis (0)]
14.  Mariette M, Meunier B, Pezet , Dalban C, Collet D, Thomas PA, Brigand C, Perniceni T, Carrere N, Bonnetain F. Hybrid minimally invasive versus open oesophagectomy for patients with oesophageal cancer: A multicenter, open-label, randomized phase III controlled trial, the MIRO trial. J Clin Oncol. 2015;3 Suppl3:Abstr5.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 44]  [Cited by in F6Publishing: 43]  [Article Influence: 4.8]  [Reference Citation Analysis (0)]
15.  Roh HF, Nam SH, Kim JM. Robot-assisted laparoscopic surgery versus conventional laparoscopic surgery in randomized controlled trials: A systematic review and meta-analysis. PLoS One. 2018;13:e0191628.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 52]  [Cited by in F6Publishing: 62]  [Article Influence: 10.3]  [Reference Citation Analysis (0)]
16.  Straatman J, van der Wielen N, Cuesta MA, Daams F, Roig Garcia J, Bonavina L, Rosman C, van Berge Henegouwen MI, Gisbertz SS, van der Peet DL. Minimally Invasive Versus Open Esophageal Resection: Three-year Follow-up of the Previously Reported Randomized Controlled Trial: the TIME Trial. Ann Surg. 2017;266:232-236.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 249]  [Cited by in F6Publishing: 334]  [Article Influence: 47.7]  [Reference Citation Analysis (0)]
17.  Guo W, Zou YB, Ma Z, Niu HJ, Jiang YG, Zhao YP, Gong TQ, Wang RW. One surgeon’s learning curve for video-assisted thoracoscopic esophagectomy for esophageal cancer with the patient in lateral position: how many cases are needed to reach competence? Surg Endosc. 2013;27:1346-1352.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 36]  [Cited by in F6Publishing: 43]  [Article Influence: 3.6]  [Reference Citation Analysis (0)]
18.  Lin J, Kang M, Chen C, Lin R, Zheng W, Zhug Y, Deng F, Chen S. Thoracolaparoscopy oesophagectomy and extensive two-field lymphadenectomy for oesophageal cancer: introduction and teaching of a new technique in a high-volume centre. Eur J Cardiothorac Surg. 2013;43:115-121.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 24]  [Article Influence: 2.0]  [Reference Citation Analysis (0)]
19.  Mu JW, Gao SG, Xue Q, Mao YS, Wang DL, Zhao J, Gao YS, Huang JF, He J. Updated experiences with minimally invasive McKeown esophagectomy for esophageal cancer. World J Gastroenterol. 2015;21:12873-12881.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in CrossRef: 35]  [Cited by in F6Publishing: 36]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
20.  Fleshman J, Marcello P, Stamos MJ, Wexner SD; American Society of Colon and Rectal Surgeons (ASCRS); Society of American Gastrointestinal and Endoscopic Surgeons (SAGES). Focus Group on Laparoscopic Colectomy Education as endorsed by The American Society of Colon and Rectal Surgeons (ASCRS) and The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES). Dis Colon Rectum. 2006;49:945-949.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 22]  [Cited by in F6Publishing: 22]  [Article Influence: 1.2]  [Reference Citation Analysis (0)]
21.  Schout BM, Hendrikx AJ, Scherpbier AJ, Bemelmans BL. Update on training models in endourology: a qualitative systematic review of the literature between January 1980 and April 2008. Eur Urol. 2008;54:1247-1261.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 73]  [Cited by in F6Publishing: 71]  [Article Influence: 4.4]  [Reference Citation Analysis (0)]
22.  Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS. Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ. 2009;338:b1802.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 381]  [Cited by in F6Publishing: 332]  [Article Influence: 22.1]  [Reference Citation Analysis (0)]
23.  Harris C, Garrubba M, Allen K, King R, Kelly C, Thiagarajan M, Castleman B, Ramsey W, Farjou D. Development, implementation and evaluation of an evidence-based program for introduction of new health technologies and clinical practices in a local healthcare setting. BMC Health Serv Res. 2015;15:575.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 20]  [Article Influence: 2.2]  [Reference Citation Analysis (0)]
24.  Gottlieb-Vedi E, Mackenzie H, van Workum F, Rosman C, Lagergren P, Markar S, Lagergren J. Surgeon Volume and Surgeon Age in Relation to Proficiency Gain Curves for Prognosis Following Surgery for Esophageal Cancer. Ann Surg Oncol. 2018; Epub ahead of print.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 12]  [Cited by in F6Publishing: 14]  [Article Influence: 2.3]  [Reference Citation Analysis (0)]
25.  Visser E, van Rossum PSN, van Veer H, Al-Naimi K, Chaudry MA, Cuesta MA, Gisbertz SS, Gutschow CA, Hölscher AH, Luyer MDP. A structured training program for minimally invasive esophagectomy for esophageal cancer- a Delphi consensus study in Europe. Dis Esophagus. 2018;31.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 12]  [Cited by in F6Publishing: 8]  [Article Influence: 1.3]  [Reference Citation Analysis (0)]
26.  Gooszen JAH, Goense L, Gisbertz SS, Ruurda JP, van Hillegersberg R, van Berge Henegouwen MI. Intrathoracic versus cervical anastomosis and predictors of anastomotic leakage after oesophagectomy for cancer. Br J Surg. 2018;105:552-560.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 70]  [Cited by in F6Publishing: 81]  [Article Influence: 13.5]  [Reference Citation Analysis (0)]
27.  McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC, Nicholl J; Balliol Collaboration, Aronson JK, Barkun JS, Blazeby JM, Boutron IC, Campbell WB, Clavien PA, Cook JA, Ergina PL, Feldman LS, Flum DR, Maddern GJ, Nicholl J, Reeves BC, Seiler CM, Strasberg SM, Meakins JL, Ashby D, Black N, Bunker J, Burton M, Campbell M, Chalkidou K, Chalmers I, de Leval M, Deeks J, Ergina PL, Grant A, Gray M, Greenhalgh R, Jenicek M, Kehoe S, Lilford R, Littlejohns P, Loke Y, Madhock R, McPherson K, Meakins J, Rothwell P, Summerskill B, Taggart D, Tekkis P, Thompson M, Treasure T, Trohler U, Vandenbroucke J. No surgical innovation without evaluation: the IDEAL recommendations. Lancet. 2009;374:1105-1112.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1181]  [Cited by in F6Publishing: 1232]  [Article Influence: 82.1]  [Reference Citation Analysis (0)]