Brief Article
Copyright ©2011 Baishideng Publishing Group Co., Limited. All rights reserved.
World J Gastroenterol. Dec 21, 2011; 17(47): 5214-5220
Published online Dec 21, 2011. doi: 10.3748/wjg.v17.i47.5214
Meta-analysis of robotic and laparoscopic surgery for treatment of rectal cancer
Shuang Lin, Hong-Gang Jiang, Zhi-Heng Chen, Shu-Yang Zhou, Xiao-Sun Liu, Ji-Ren Yu
Shuang Lin, Hong-Gang Jiang, Zhi-Heng Chen, Department of Oncological Surgery, First Affiliated Hospital of Jiaxing University, Jiaxing 314000, Zhejiang Province, China
Shu-Yang Zhou, Xiao-Sun Liu, Ji-Ren Yu, Department of Gastroenterological Surgery, First Affiliated Hospital, Medical College, Zhejiang University, Hangzhou 310003, Zhejiang Province, China
Author contributions: Lin S wrote the paper and performed the research; Yu JR and Jiang HG designed the research; Liu XS and Chen ZH collected the data; Chen ZH and Zhou SY performed a literature search and retrieved the data; Yu JR was consulted and analyzed the data.
Supported by The National Natural Science Foundation of China, No. 81071964; Zhejiang Provincial Natural Science Foundation of China, No. Y2110019
Correspondence to: Ji-Ren Yu, MD, Department of Gastroenterological Surgery, First Affiliated Hospital, Medical College, Zhejiang University, Hangzhou 310003, Zhejiang Province, China. yujiren0909@hotmail.com
Telephone: +86-571-87236147 Fax: +86-571-87072577
Received: August 11, 2011
Revised: October 2, 2011
Accepted: November 9, 2011
Published online: December 21, 2011

Abstract

AIM: To conduct a meta-analysis to determine the relative merits of robotic surgery (RS) and laparoscopic surgery (LS) for rectal cancer.

METHODS: A literature search was performed to identify comparative studies reporting perioperative outcomes for RS and LS for rectal cancer. Pooled odds ratios and weighted mean differences (WMDs) with 95% confidence intervals (95% CIs) were calculated using either the fixed effects model or random effects model.

RESULTS: Eight studies matched the selection criteria and reported on 661 subjects, of whom 268 underwent RS and 393 underwent LS for rectal cancer. Compared the perioperative outcomes of RS with LS, reports of RS indicated favorable outcomes considering conversion (WMD: 0.25; 95% CI: 0.11-0.58; P = 0.001). Meanwhile, operative time (WMD: 27.92, 95% CI: -13.43 to 69.27; P = 0.19); blood loss (WMD: -32.35, 95% CI: -86.19 to 21.50; P = 0.24); days to passing flatus (WMD: -0.18, 95% CI: -0.96 to 0.60; P = 0.65); length of stay (WMD: -0.04; 95% CI: -2.28 to 2.20; P = 0.97); complications (WMD: 1.05; 95% CI: 0.71-1.55; P = 0.82) and pathological details, including lymph nodes harvested (WMD: 0.41, 95% CI: -0.67 to 1.50; P = 0.46), distal resection margin (WMD: -0.35, 95% CI: -1.27 to 0.58; P = 0.46), and positive circumferential resection margin (WMD: 0.54, 95% CI: 0.12-2.39; P = 0.42) were similar between RS and LS.

CONCLUSION: RS for rectal cancer is superior to LS in terms of conversion. RS may be an alternative treatment for rectal cancer. Further studies are required.

Key Words: Robotic surgery, Laparoscopic surgery, Rectal cancer, Da Vinci robotic system, Meta-analysis



INTRODUCTION

Over the past 30 years, laparoscopic surgery (LS) has revolutionized general surgical practice, above all affecting surgery of the gastrointestinal (GI) tract[1,2]. However, with regard to rectal cancer, there are several technical drawbacks to LS, including limited range of motion of instruments in a narrow pelvic cavity, related loss of dexterity, and an inadequate visual field associated with unstable camera view and assistant’s traction, which are not under the surgeon’s control[3]. Technical advantages of the Da Vinci robotic system could overcome the limitations of LS for rectal cancer, by giving the surgeon a 3D view, better ergonomics, enhanced dexterity, precision and control due to the 3D optical system and EndoWrist® Instruments.

Although surgical robots have been successfully applied to a number of disciplines, most notably urological and cardiac procedures[4,5], robotic rectal surgery remains in its infancy. Most studies have been limited by small sample size and a single institution design. To overcome these limitations, a meta-analysis of studies comparing robotic surgery (RS) and LS for rectal cancer should be performed. The aim of this meta-analysis was to determine the relative merits of RS and LS for rectal cancer.

MATERIALS AND METHODS
Study selection

The Pubmed, Embase, Cochrane Library, Ovid, and Web of Science databases were searched systematically for all articles published before June 2011 to compare RS and LS for rectal cancer. The terms used for the search were: “robotic” and “rectal cancer”. Only studies in the English language were considered for inclusion. Reference lists of all retrieved articles were manually searched for additional studies. Two reviewers independently extracted the data from each study. All relevant text, tables and figures were reviewed for data extraction. Discrepancies between the two reviewers were resolved by discussion and consensus.

Criteria for inclusion and exclusion

For inclusion in the meta-analysis, a study had to fulfill the following criteria: (1) compare the outcomes of RS and LS, regardless of other diseases; (2) report on at least one of the outcome measures mentioned below; and (3) if dual (or multiple) studies were reported by the same institution and/or authors, either the one of higher quality or the most recent publication was included in the analysis.

Abstracts, letters, editorials and expert opinions, reviews without original data, case reports and studies lacking control groups were excluded. The following studies or data were also excluded: (1) the outcomes and parameters of patients were not clearly reported (e.g., with no clearly reported outcomes of SD; (2) it was impossible to extract the appropriate data from the published results; and (3) there was overlap between authors or centers in the published literature.

Outcomes of interest

The following outcomes were used to compare the two operative techniques: (1) intraoperative data, which included operative time, blood loss and conversion; (2) postoperative data, which included complication, days to passing flatus, and length of stay; and (3) pathological details, which included lymph nodes harvested, distal resection margin (DRM), and positive circumferential resection margin (PCRM) which was defined as a circumferential resection margin (CRM) of ≤ 1 mm.

Data extraction

Two reviewers independently extracted the following parameters from each study: (1) first author and year of publication; (2) study population characteristics; (3) number of subjects operated on with each technique; and (4) intraoperative data, postoperative data, and pathological details.

Statistical analysis

The meta-analysis was performed using the Review Manager (RevMan) software, version 4.2.2. We analyzed dichotomous variables using estimation of odds ratios with a 95% confidence interval (95% CI) and continuous variables using weighted mean difference (WMD) with a 95% CI. Pooled effect was calculated using either the fixed effects model or random effects model. Heterogeneity was evaluated by χ2 and I2. We considered heterogeneity to be present if the I2 statistic was > 50%. P < 0.05 was considered significant.

RESULTS
Selection of trials

The initial search strategy retrieved 154 publications, after screening all titles, abstracts, and full-text. A total of eight studies met our entry criteria and were retrieved for more detailed evaluation. The characteristics of these eight studies are summarized in Table 1[6-13]. Eight studies [six non-randomized controlled trials (NRCTs), two randomized controlled trials (RCTs)] included a total of 661 patients: 268 in the RS group and 393 in the LS group. Two studies were conducted in United States[7,13], three in Korea[6,8,12], two in Italy[10,11], and one in Romania[9]. The sample size of each study varied from six to 123 patients. In the included studies, six were considered as level of evidence 3, and the remaining 2 as level of evidence 2 (according to the grading of the Centre of Evidence-Based Medicine, Oxford, United Kingdom; http://www.cebm.net/index.aspx?o =5653).

Table 1 Characteristics of included studies.
StudyCountryGroupNo. of patientsMean age (yr)Gender (M/F)Level of evidence
Park et al[6]KoreaRS5257.3 (± 12.3)28/243
LS12365.1 (± 10.3)70/53
Baek et al[7]United StatesRS4163.6 (48-87)25/163
LS4163.7 (42-88)25/16
Kwak et al[8]KoreaRS5960 (53-68)39/203
LS5959 (53-69)42/17
Popescu et al[9]RomaniaRS3853 (± 11.27)23/153
LS8460 (± 12.27)51/33
Bianchi et al[10]ItalyRS2569 (33-83)18/72
LS2562 (42-77)17/8
Patriti et al[11]ItalyRS2968 ± 10NA3
LS3769 ± 10NA
Baik et al[12]KoreaRS1857.3 ± 6.314/42
LS1862.0 ± 9.014/4
Pigazzi et al[13]United StatesRS660 (42-78)4/23
LS670 (57-88)2/4

In these studies, patients in the two groups were matched for operation time[6,9,11,12], blood loss[9,11], conversion[6-13], complications[6-13], days to passing flatus[6,12], length of stay[6,11,12], lymph nodes harvested[6,9,11,12], DRM[6,11,12], and PCRM[6,7,10].

Meta-analysis of intraoperative data

In four studies, operative time showed that there was no significant difference between the two groups. Analysis of the pooled data revealed that the two groups did not differ significantly in this regard (WMD: 27.92, 95% CI: -13.43 to 69.27; P = 0.19) (Figure 1A).

Figure 1
Figure 1 Forest plot displaying the results of the meta-analysis on operative time (A), blood loss (B), conversion (C), days to passing flatus (D), length of stay (E), complications (F), lymph nodes harvested (G), distal resection margin (H) and positive circumferential resection margin (I). RS: Robotic surgery; LS: Laparoscopic surgery; OR: Odds ratio; WMD: Weighted mean difference.

In two studies, blood loss did not differ significantly between the two groups. Analysis of the pooled data revealed that the two groups did not differ significantly in this regard (WMD: -32.35, 95% CI: -86.19 to 21.50; P = 0.24) (Figure 1B).

In all eight studies, conversion was found to be significantly lower in the RS group than in the LS group. Moreover, analysis of the pooled data revealed that conversion for RS was significantly lower by 0.25-fold (WMD: 0.25; 95% CI: 0.11-0.58; P = 0.001) (Figure 1C).

Meta-analysis of postoperative outcomes

In two studies, number of days to passing flatus was significantly lower in the RS group vs the LS group. Meanwhile, analysis of the pooled data revealed that the two groups did not differ significantly in this regard (WMD: -0.18, 95% CI: -0.96 to 0.60; P = 0.65) (Figure 1D).

In three studies, length of stay was found to be no different in the RS group and the LS group. Meanwhile, analysis of the pooled data revealed that the two groups did not differ significantly in this regard (WMD: -0.04; 95% CI: -2.28 to 2.20; P = 0.97) (Figure 1E).

In all eight studies, complications were found to be no different in the RS group and the LS group. Meanwhile, analysis of the pooled data revealed that the two groups did not differ significantly in this regard (WMD: 1.05; 95% CI: 0.71-1.55; P = 0.82) (Figure 1F).

Meta-analysis of pathological details

In the four studies, lymph nodes harvested showed that there was no significant difference between the two groups. Analysis of the pooled data revealed that the two groups did not differ significantly in this regard (WMD: 0.41, 95% CI: -0.67 to 1.50; P = 0.46) (Figure 1G).

In three studies, DRM was found to be significantly lower in the RS group than the LS group. Meanwhile, analysis of the pooled data revealed that the two groups did not differ significantly in this regard (WMD: -0.35, 95% CI: -1.27 to 0.58; P = 0.46) (Figure 1H).

In three studies, PCRM showed that there was no significant difference between the two groups. Analysis of the pooled data revealed that the two groups did not differ significantly in this regard (WMD: 0.54, 95% CI: 0.12-2.39; P = 0.42) (Figure 1I).

Heterogeneity

A significant heterogeneity was recognized in the following two factors: operative time, blood loss, days to passing flatus, length of stay and DRM.

DISCUSSION

Meta-analysis could be used to evaluate the existing li-terature in both qualitative and quantitative ways by comparing and integrating the results of different studies and taking into account variations in characteristics that could influence the overall estimate of the outcome of interest[14]. Although meta-analysis has traditionally been applied and best confined to RCTs, meta-analytical techniques using NRCTs might be a good method in some clinical settings in which either the number or the sample size of RCTs was insufficient[15]. To the best of our knowledge, this was the first comprehensive meta-analysis comparing RS versus LS for rectal cancer.

RS is often perceived as being more time-consuming, because of the additional set-up time required[16]. It usually requires two steps for rectal cancer[17,18]. After dissection of the left colon and sigmoid colon and division and ligation of the inferior mesenteric vessels, the da Vinci system must be moved for the next step. However, moving the da Vinci system is a time-consuming and difficult procedure because the robotic devices are heavy and bulky. This meta-analysis revealed that there was no significant difference in operative time between RS and LS. This finding could be attributable to the shortened learning curve, and it has been suggested that the intuitive controls of robotic systems, more comparable with open surgery, could shorten the learning curve, even in the hands of relatively inexperienced laparoscopic surgeons[19]. As we overcame the learning curve with experience and prevented collisions by properly positioning the robotic ports, the operation time decreased. There was no significant difference in blood loss when comparing RS and LS.

Conversion to open surgery and complications are critical in minimally invasive rectal cancer surgery, because converted patients have higher complication rates[20] and, in one series at least, worse oncological outcomes[21]. Conversion rate was significantly lower in the RS group than in the LS group. Lower conversion with RS might have been due to superior exposure and visualization of the operating field in the pelvis, thanks to the ability of the fixed fourth arm to grip and maneuver organs; the ability of the surgeon to move the 3D camera as required; and the greater ease of dissection afforded by the highly maneuverable EndoWrist instruments attached to the robotic arms.

Number of days to passing flatus was lower in the RS group than the LS group, meanwhile, length of stay was found to be no different between the two groups. However, analysis of the pooled data did not reveal any significant difference in this regard. These findings implied that the time required for patients to resume daily activities might not be shorter after RS than LS. There was no significant difference in complications when comparing RS and LS. On the contrary, it has been postulated that these characteristics of RS could make patients recover faster and reduce complications, because with the da Vinci surgical system, robotic arms are used for retraction and dissection during the total mesorectal excision procedure, and their use reduces unnecessary procedures and minimizes iatrogenic tissue injury during retraction. These findings are difficult to explain, and more advanced studies are needed before such conclusions can be drawn.

We postulated that specimen quality could be used as an indicator to predict long-term clinical oncological results. No significant differences were proved between RS and LS in the pathological details, including harvested lymph nodes, DRM and PCRM. The number of harvested lymph nodes, DRM, and PCRM did not differ significantly between the two groups in our meta-analysis. This demonstrated that RS could be performed safely and with a high success rate following oncological principles compared with LS. However, long-term follow-up evaluation is necessary to evaluate the exact oncological outcomes of RS for rectal cancer.

The cost of RS equipment is very high and likely to be a serious impediment to uptake in the foreseeable future[22]. However, it is important to perform a cost-effectiveness analysis between RS and LS. Only one trial has reported that the average total hospitalization costs were higher in the RS group ($83 915) than in the LS group ($62 601), and these differences were not statistically significant[7]. To the best of our knowledge, total hospitalization costs may be due to the greater expense and consumption of operating room resources such as space and the availability of skilled technical staff, and differ significantly between hospitals[23]. Therefore, insufficient data and great heterogeneity precluded a meta-analysis of cost-effectiveness.

Significant heterogeneity in those articles was observed in the operative time, blood loss, days to passing flatus, length of stay and DRM, which may be explained by the difference in skill, extension of lymph node dissection, and duration of learning curve. Regarding the heterogeneity between the articles, random-effect models were used in this meta-analysis.

The results of the present meta-analysis should be interpreted with caution because of several limitations. First, some data came from NRCTs, and the overall level of clinical evidence was low. It has been reported that NRCTs can either exaggerate or underestimate the magnitude of measured effects in a study of intervention regardless of quality scores[24]. However, Abrahama et al[25] have found that meta-analysis of well-designed NRCTs of surgical procedures was probably as accurate as that of RCTs. In fact, six studies included in the present study were NRCTs. Second, there was heterogeneity between the two groups, because it was impossible to match patient characteristics in all studies. We applied a random-effect model to take into consideration between-study variation, and it might have been expected to exert a limited influence. Finally, authors might be more likely to report positive results, and studies with significant outcomes were more likely to be published, so potential publication bias might have been present.

In conclusion, the results of this meta-analysis of 661 patients show that RS is superior to LS for rectal cancer in terms of conversion. Therefore, RS may be an alternative treatment for rectal cancer. Further studies are required to better define its role.

COMMENTS
Background

The da Vinci robotic system was introduced as the next advance in minimally invasive surgery to overcome the technical limitations of laparoscopy, but robotic rectal surgery is controversial because of a lack of well-powered trials.

Research frontiers

Meta-analysis was used to evaluate the relative merits of robotic surgery (RS) and laparoscopic surgery (LS) for rectal cancer in this study.

Innovations and breakthroughs

The meta-analysis reported that RS had favorable outcomes considering conversion, compared with LS for rectal cancer. Meanwhile, operative time, blood loss, days to passing flatus, length of stay, complications and pathological details, including lymph nodes harvested, distal resection margin, and positive circumferential resection margin were similar between RS and LS. This is believed to be the first comprehensive meta-analysis comparing RS and LS for rectal cancer.

Applications

The results of this meta-analysis show that RS is superior to LS in terms of conversion. Therefore, RS may be an alternative treatment for rectal cancer.

Peer review

This paper addressed superiority of RS for rectal cancer, especially due to superior exposure and visualization of the intrapelvic field. This advantage means that surgeons complete the operation without conversion. This paper should be of interest to colorectal surgeons worldwide.

Footnotes

Peer reviewers: Keiji Hirata, MD, Surgery 1, University of Occupational and Environmental Health, 1-1 Iseigaoka, Yahatanishi-ku, Kitakyushu 807-8555, Japan; Dr. Kok Sun Ho, Department of Colorectal Surgery, Singapore General Hospital, Outram Road, Singapore 169608, Singapore; Yik-Hong Ho, Professor, Department of Surgery, School of Medicine, James Cook University, Townsville 4811, Australia

S- Editor Tian L L- Editor Kerr C E- Editor Zhang DN

References
1.  Bai HL, Chen B, Zhou Y, Wu XT. Five-year long-term outcomes of laparoscopic surgery for colon cancer. World J Gastroenterol. 2010;16:4992-4997.  [PubMed]  [DOI]
2.  Bartels SA, Vlug MS, Ubbink DT, Bemelman WA. Quality of life after laparoscopic and open colorectal surgery: a systematic review. World J Gastroenterol. 2010;16:5035-5041.  [PubMed]  [DOI]
3.  Cadière GB, Himpens J, Germay O, Izizaw R, Degueldre M, Vandromme J, Capelluto E, Bruyns J. Feasibility of robotic laparoscopic surgery: 146 cases. World J Surg. 2001;25:1467-1477.  [PubMed]  [DOI]
4.  Menon M, Tewari A, Baize B, Guillonneau B, Vallancien G. Prospective comparison of radical retropubic prostatectomy and robot-assisted anatomic prostatectomy: the Vattikuti Urology Institute experience. Urology. 2002;60:864-868.  [PubMed]  [DOI]
5.  Tatooles AJ, Pappas PS, Gordon PJ, Slaughter MS. Minimally invasive mitral valve repair using the da Vinci robotic system. Ann Thorac Surg. 2004;77:1978-1982; discussion 1978-1982;.  [PubMed]  [DOI]
6.  Park JS, Choi GS, Lim KH, Jang YS, Jun SH. S052: a comparison of robot-assisted, laparoscopic, and open surgery in the treatment of rectal cancer. Surg Endosc. 2011;25:240-248.  [PubMed]  [DOI]
7.  Baek JH, Pastor C, Pigazzi A. Robotic and laparoscopic total mesorectal excision for rectal cancer: a case-matched study. Surg Endosc. 2011;25:521-525.  [PubMed]  [DOI]
8.  Kwak JM, Kim SH, Kim J, Son DN, Baek SJ, Cho JS. Robotic vs laparoscopic resection of rectal cancer: short-term outcomes of a case-control study. Dis Colon Rectum. 2011;54:151-156.  [PubMed]  [DOI]
9.  Popescu I, Vasilescu C, Tomulescu V, Vasile S, Sgarbura O. The minimally invasive approach, laparoscopic and robotic, in rectal resection for cancer. A single center experience. Acta Chir Iugosl. 2010;57:29-35.  [PubMed]  [DOI]
10.  Bianchi PP, Ceriani C, Locatelli A, Spinoglio G, Zampino MG, Sonzogni A, Crosta C, Andreoni B. Robotic versus laparoscopic total mesorectal excision for rectal cancer: a comparative analysis of oncological safety and short-term outcomes. Surg Endosc. 2010;24:2888-2894.  [PubMed]  [DOI]
11.  Patriti A, Ceccarelli G, Bartoli A, Spaziani A, Biancafarina A, Casciola L. Short- and medium-term outcome of robot-assisted and traditional laparoscopic rectal resection. JSLS. 2009;13:176-183.  [PubMed]  [DOI]
12.  Baik SH, Ko YT, Kang CM, Lee WJ, Kim NK, Sohn SK, Chi HS, Cho CH. Robotic tumor-specific mesorectal excision of rectal cancer: short-term outcome of a pilot randomized trial. Surg Endosc. 2008;22:1601-1608.  [PubMed]  [DOI]
13.  Pigazzi A, Ellenhorn JD, Ballantyne GH, Paz IB. Robotic-assisted laparoscopic low anterior resection with total mesorectal excision for rectal cancer. Surg Endosc. 2006;20:1521-1525.  [PubMed]  [DOI]
14.  Aziz O, Constantinides V, Tekkis PP, Athanasiou T, Purkayastha S, Paraskeva P, Darzi AW, Heriot AG. Laparoscopic versus open surgery for rectal cancer: a meta-analysis. Ann Surg Oncol. 2006;13:413-424.  [PubMed]  [DOI]
15.  Mathurin P, Raynard B, Dharancy S, Kirzin S, Fallik D, Pruvot FR, Roumilhac D, Canva V, Paris JC, Chaput JC. Meta-analysis: evaluation of adjuvant therapy after curative liver resection for hepatocellular carcinoma. Aliment Pharmacol Ther. 2003;17:1247-1261.  [PubMed]  [DOI]
16.  D'Annibale A, Morpurgo E, Fiscon V, Trevisan P, Sovernigo G, Orsini C, Guidolin D. Robotic and laparoscopic surgery for treatment of colorectal diseases. Dis Colon Rectum. 2004;47:2162-2168.  [PubMed]  [DOI]
17.  Choi DJ, Kim SH, Lee PJ, Kim J, Woo SU. Single-stage totally robotic dissection for rectal cancer surgery: technique and short-term outcome in 50 consecutive patients. Dis Colon Rectum. 2009;52:1824-1830.  [PubMed]  [DOI]
18.  Koh DC, Tsang CB, Kim SH. A new application of the four-arm standard da Vinci® surgical system: totally robotic-assisted left-sided colon or rectal resection. Surg Endosc. 2011;25:1945-1952.  [PubMed]  [DOI]
19.  Ng SS, Lee JF, Yiu RY, Li JC, Hon SS. Telerobotic-assisted laparoscopic abdominoperineal resection for low rectal cancer: report of the first case in Hong Kong and China with an updated literature review. World J Gastroenterol. 2007;13:2514-2518.  [PubMed]  [DOI]
20.  Guillou PJ, Quirke P, Thorpe H, Walker J, Jayne DG, Smith AM, Heath RM, Brown JM. Short-term endpoints of conventional versus laparoscopic-assisted surgery in patients with colorectal cancer (MRC CLASICC trial): multicentre, randomised controlled trial. Lancet. 2005;365:1718-1726.  [PubMed]  [DOI]
21.  Rottoli M, Bona S, Rosati R, Elmore U, Bianchi PP, Spinelli A, Bartolucci C, Montorsi M. Laparoscopic rectal resection for cancer: effects of conversion on short-term outcome and survival. Ann Surg Oncol. 2009;16:1279-1286.  [PubMed]  [DOI]
22.  Markar SR, Karthikesalingam AP, Hagen ME, Talamini M, Horgan S, Wagner OJ. Robotic vs. laparoscopic Nissen fundoplication for gastro-oesophageal reflux disease: systematic review and meta-analysis. Int J Med Robot. 2010;6:125-131.  [PubMed]  [DOI]
23.  Zhang P, Tian JH, Yang KH, Li J, Jia WQ, Sun SL, Ma B, Liu YL. Robot-assisted laparoscope fundoplication for gastroesophageal reflux disease: a systematic review of randomized controlled trials. Digestion. 2010;81:1-9.  [PubMed]  [DOI]
24.  MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AM. A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technol Assess. 2000;4:1-154.  [PubMed]  [DOI]
25.  Abraham NS, Byrne CJ, Young JM, Solomon MJ. Meta-analysis of well-designed nonrandomized comparative studies of surgical procedures is as good as randomized controlled trials. J Clin Epidemiol. 2010;63:238-245.  [PubMed]  [DOI]