Evidence-Based Medicine Open Access
Copyright ©The Author(s) 2020. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastrointest Oncol. Jan 15, 2020; 12(1): 113-123
Published online Jan 15, 2020. doi: 10.4251/wjgo.v12.i1.113
Validity of studies suggesting preoperative chemotherapy for resectable thoracic esophageal cancer: A critical appraisal of randomized trials
Giulia Manzini, Ursula Klotz, Doris Henne-Bruns, Michael Kremer, Department of General and Visceral Surgery, University of Ulm, Ulm 89081, Germany
Michael Kremer, Department of General and Visceral Surgery, Hospital of Aarau, Aarau 5000, Switzerland
ORCID number: Giulia Manzini (0000-0002-8032-8043); Ursula Klotz (0000-0003-2725-1187); Doris Henne-Bruns (0000-0002-5699-9031); Michael Kremer (0000-0001-9364-9420).
Author contributions: Manzini G, Klotz U and Henne-Bruns DH contributed substantially to the conception and design of the study; Manzini G and Klotz U contributed to the analysis and interpretation of the data. Manzini G performed the meta-analysis. Manzini G and Kremer M wrote the manuscript. All authors gave their final approval of the version to be published.
Conflict-of-interest statement: The authors report no relevant conflicts of interest.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Corresponding author: Doris Henne-Bruns, MD, Professor, Department of General and Visceral Surgery, University Hospital of Ulm, Albert-Einstein-Allee 23, Ulm 89081, Germany. doris.henne-bruns@uniklinik-ulm.de
Received: March 22, 2019
Peer-review started: March 22, 2019
First decision: July 31, 2019
Revised: September 20, 2019
Accepted: October 14, 2019
Article in press: October 14, 2019
Published online: January 15, 2020

Abstract
BACKGROUND

In 2015, Kidane published a Cochrane review and meta-analysis to summarise the impact of preoperative chemotherapy versus surgery alone on survival for resectable thoracic esophageal cancer. The authors concluded that preoperative chemotherapy improved overall survival (OS).

AIM

The aim of this article was to assess the validity of the three most powerful studies included in the Cochrane review and the meta-analysis supporting the advantage of preoperative chemotherapy and to investigate the impact of an exclusion of these three studies on the result of the meta-analysis.

METHODS

OS was selected as the endpoint of interest. Among the ten included papers which analysed this endpoint, we identified the three publications with the highest weights influencing the final result. The validity of these papers was analysed using the CONSORT checklist for randomized controlled trials. We performed a new meta-analysis without the three studies to assess their impact on the general result of the original meta-analysis.

RESULTS

The three analysed studies revealed several inconsistencies. Inappropriate answers were found in up to one third of the items of the CONSORT checklist. Missing information about sample-size calculation and power, unclear or inadequate randomisation, and missing blinded set-up were the most common findings. When the three criticized studies were excluded in the meta-analysis, preoperative chemotherapy showed no benefit in OS.

CONCLUSION

The three most powerful publications in the Cochrane review show substantial deficits. After the exclusion of these studies from the meta-analysis, preoperative chemotherapy does not seem to result in an advantage in survival. We suggest a more critical appraisal regarding the validity of single studies.

Key Words: Esophageal cancer, Assessment of validity, Meta-analysis, CONSORT, Overall survival

Core tip: The quality of single studies is crucial in order to perform valid meta-analyses that are often used as basis for guideline recommendations. We critically analysed a recent Cochrane meta-analysis that supports the use of preoperative chemotherapy for resectable thoracic esophageal cancer in order to improve overall survival. The most powerful included studies showed several inconsistencies according to the requirements of the Consort checklist for randomized controlled trials. After the exclusion of these studies from the meta-analysis, preoperative chemotherapy does not seem to result in an advantage in survival. We suggest a more critical appraisal regarding the validity of single studies.



INTRODUCTION

Esophageal cancer is the eighth most common cancer in the world and the sixth most common cause of death from cancer with an overall ratio of mortality of 0.88[1]. Although it accounts for only 3.2% of all cancers, the incidence of esophageal cancer is increasing with an incidence of 572/100000 new cases/year in 2018[2].

Surgery is the treatment of choice for localized esophageal cancer[3-4] with a potential to provide loco-regional control, as well as long-term survival[5]. Curative resection is possible in only 15% to 39% of the cases[6-9]. Surgery is the only curative treatment, but it alone often fails to overcome the natural history of the disease owing to the presence of occult micrometastases, and fatal distant and loco-regional disease relapse is common. Median survival after esophagectomy with curative intention is 15 to 18 mo with a 5-year survival rate of 20% to 25%[5]. Therefore, clinicians are now inclined to use some form of multidisciplinary treatment including surgery as a standard of care for locally advanced esophageal cancer, which is defined as disease restricted to the esophagus or resectable periesophageal tissue (T2-T4) and/or lymph-node involvement (N1-N3) in the absence of distant metastasis[10].

The optimal multimodality treatment is still controversial. Potential contentious issues exist regarding the (1) ideal preoperative, perioperative or postoperative approach and (2) ideal combination of radiotherapy (RTx), chemotherapy (CTx) or concurrent chemoradiation. Various randomized and non-randomized trials and several meta-analyses have been conducted to address this topic, but established standard guidelines still vary considerably or even fail to propose a specific treatment regime[11]. Preoperative (radio-)chemotherapy aims to exterminate micro-metastases, enhance resectability by down-staging the tumour, improve loco-regional control and provide relief of dysphagia[11,12].

Several studies have investigated whether preoperative CTx leads to improved cure rates, but reports remain conflicting. The initial Cochrane review of preoperative CTx for resectable esophageal cancer[13] concluded that no survival advantage was associated with CTx. The same result was found by Urschel et al[14] after inclusion of 11 randomized trials in a meta-analysis. Ychou et al[15], Boonstra et al[16] and MRC Allum et al[17] subsequently reported a survival benefit for patients receiving neoadjuvant CTx. After inclusion of these last three studies, the updated Cochrane Review and meta-analysis by Kidane et al[18] on the same topic found an improvement in overall survival (OS) [hazard ratio (HR): 0.88; 95% confidence interval (CI): 0.80 to 0.96] for patients receiving preoperative CTx. A total of ten randomized controlled studies with OS as the primary endpoint were included in this meta-analysis.

The aim of our study was to assess the validity of the studies by Ychou et al[15], Boonstra et al[16] and MRC Allum et al[17] included in the updated Cochrane Review and the meta-analysis by Kidane et al[18], which confirmed the benefit of preoperative CTx on survival for resectable thoracic esophageal cancer with the intention to invite everyone to critically interpret not only the results, but also the methodology by which the results were achieved. We performed a variety of meta-analyses excluding or including studies depending on their validity and attributed power and discuss those findings in regard to current recommendations of esophageal carcinoma guidelines.

MATERIALS AND METHODS

The meta-analysis by Kidane et al[18] included a total of ten studies. Four (40%) (Ychou et al[15], Boonstra et al[16], MRC Allum et al[17], and Law et al[19]) found a statistically significant advantage in survival in patients after preoperative CTx for resectable thoracic esophageal cancer (HR: < 1 with a significant 95%CI). All the other six included studies (60%) were not statistically significant[20-25].

In the first part of the results section we assessed the validity of the three most powerful studies included in the Cochrane review by Kidane et al[18], which found a statistically significant advantage in survival in patients receiving preoperative CTx before resection for thoracic esophageal cancer. These studies were those of Ychou et al[15], Boonstra et al[16] and MRC Allum et al[17].

In the second part of the results section, we performed a new meta-analysis without these aforementioned three studies. Among the three analysed studies, Boonstra et al[16] had the higher validity, so we performed another meta-analysis assuming that this study is valid enough to be included in the meta-analysis.

Finally, we present the results of the meta-analysis excluding the four statistically significant studies confirming the survival advantage for patients treated with preoperative CTx. In this last case, only statistically non-significant studies were included in the meta-analysis.

Selection of the studies and assessment of their validity

We used the same methodology as described in our previous publication[26] to analyse the validity of the Cochrane review. From the several endpoints investigated in the Cochrane review by Kidane et al[18], we identified OS as a major endpoint of interest. Among the ten studies identified by the authors of the Cochrane review investigating OS, we selected the three most powerful studies as weighted by the review’s authors which support the advantage of preoperative CTx: Ychou et al[15], Boonstra et al[16] and MRC Allum et al[17]. The weights assigned to these three studies by the authors of the systematic review according to their sample size, precision of the estimates and width of the confidence intervals were 24.5%, 24.1% and 20.5%, respectively. We then assessed the validity of these studies using the CONSORT checklist 2010[27], which is a validated instrument for the evaluation of randomized controlled trials (RCTs) and contains a total of 37 items. The checklist with all items and their precise description is available in the Appendix of our previous publication[26]. We then asked whether the positive result in the Cochrane review is supported by sufficient validity. Figure 1 illustrates our methodology. Two independent review authors (UK and GM) assessed the validity of each of the three publications.

Figure 1
Figure 1 Four steps to the analysis of validity of a systematic review according to our previous work[26]. We identified the endpoint of interest (overall survival) and selected the three most powerful studies addressing this endpoint on the basis of the assigned weights by the authors of the systematic review, as these studies contributed essentially to the positive result of the systematic review. We finally assessed the validity of these studies by using the CONSORT checklist.
Meta-analysis

We repeated the meta-analysis without the three analysed studies (n = 7) and compared the result with the original meta-analysis comprising ten studies. Since Boonstra et al[16] has the higher validity among the analysed studies, we then conducted a second meta-analysis only excluding Ychou et al[15] and MRC Allum et al[17]. In a next step, we assumed that all single studies with a statistically significant benefit of preoperative CTx for thoracic resectable esophageal cancer (n = 4) were not valid enough and performed a second meta-analysis with the remaining six studies. The results were compared with the original meta-analysis (n = 10 studies). The meta-analyses were performed with R, version 3.2.0, with the package “meta” (http:// www.r-project.org/foundation).

RESULTS
Assessment of the validity of the studies

Table 1 presents a summary of the three analysed papers. The results are reported for each of the three included studies. Table 2 summarizes all the items present in the CONSORT checklist showing how the studies dealt with them. In this section, we describe the problems of each study. Eleven of the 34 validity criteria (32.4%) were not met in the study by Ychou et al[15]. Three items were not applicable. The randomisation occurred by phone call through a centralized randomisation system, and then the assignment was stratified according to centre, performance status, and tumour site using the minimisation procedure. Due to the use of the minimisation method, allocation concealment was not maintained. Blinding was not possible in this study, as the control group did not receive any preoperative treatment. Inclusion of untreated controls limits the interpretation of the study. Specifically, the difference between the intervention and control group may be caused by a non-specific effect, such as a placebo effect. Moreover, 50% of the patients in the intervention group also received postoperative CTx. Regarding sample size, in the methods section the authors described that 250 patients (178 deaths) were required to achieve the needed power. The trial was closed earlier due to difficulties in patient recruitment. At the closure time, a total of 224 patients (156 deaths) had been included, raising the question of whether the power was sufficient. Moreover, patients with stomach adenocarcinoma were also included in the study at a later time after changing the inclusion criteria. Taken together, these issues lead to insufficient validity of the report; therefore the described effect cannot be considered as clinically relevant.

Table 1 Summary of the three analysed studies.
Study (year)Boonstra et al[16]MRC Allum et al[17]Ychou et al[15]
Number of included patients (intervention vs control)85 vs 84400 vs 402113 vs 111
Inclusion criteria100% squamous-cell cancer of thoracic oesophagus (upper, middle and lower third), T1-3, any N, M0 (M1a eligible if distal oesophageal cancer and suspected celiac nodes) < 80 yr of age, Karnofsky > 70Squamous-cell cancer, adenocarcinoma, undifferentiated, upper, middle and lower thirds of oesophagus, as well as the gastric cardiaResectable adenocarcinoma of the lower third of the oesophagus or gastro-oesophageal junction or stomach 18-75 years of age, WHO performance status 0 or 1, adequate renal (Cr < 120 mol/L) and hematologic functions
Intervention groupPreop. CTxa: Cisplatin, Etoposid iv. po. + surgeryPreop.CTx: Cisplatin, 5-FU + preop.radiotherapy + surgeryPreop.CTx: 5-FU, Cisplatin + surgery
Control groupSurgeryPreop. radiotherapy + surgerySurgery
Outcome (intervention vs control)Median overall survival 16 mo vs 12 mo, P = 0.03, by the log-rank test, HRb: 0.71; (95%CIc: 0.51-0.98)Overall survival is significantly greater in CS group (HR: 0.84, 95%CI: 0.72-0.98, P = 0.03)Overall survival significantly higher in CS group (HR for death 0.69, 95%CI: 0.50-0.95, P = 0.02) 5-year survival: 38% (95%CI: 29%-47%) in the CS group vs 24% (95%CI: 26%-44%) in the S group
Weight assigned in the Cochrane review (%)24.120.524.5
Table 2 Assessment of validity of the three analysed studies according to the CONSORT checklist (REF).
Section/TopicItem numberBoonstra et al[16]MRC Allum et al[17]Ychou et al[15]
Title and Abstract1aYesYesNo
1bYesYesYes
Introduction
Background and objectives2aYesYesYes
2bYesYesYes
Methods
Trial design3aYesYesYes
3bNot applicableNot applicableYes
Participants4aYesYesYes
4bYesNoNo
Interventions5YesNoYes
Outcomes6aYesYesYes
6bNot applicableNot applicableNot applicable
Sample size7aYesNoYes
7bNot applicableNot applicableYes
Randomisation
-Sequence generation8aNoYesYes
8bNoNoNo
-Allocation concealment mechanism9NoNoNo
- Implementation10NoNoNo
Blinding11aNoYesNo
11bYesNoNo
Statistical methods12aYesYesYes
12bYesYesNot applicable
Results
Participant flow13aYesYesYes
13bYesYesYes
Recruitment14aYesYesYes
14bNot applicableNot applicableNes
Baseline data15YesYesYes
Numbers analysed16YesYesYes
Outcomes and estimation17aYesYesYes
17bYesYesYes
Ancillary analysis18YesYesNot applicable
Harms19YesNoYes
Discussion
Limitations20YesYesYes
Generalisability21NoNoNo
Interpretation22YesYesYes
Other information
Registration23NoNoNo
Protocol24NoNoNo
Funding25YesYesNo

In Boonstra et al[16], we identified poor validity in 8 of the 33 validity criteria (24.2%). Four items were not applicable. Again, as in the previous study, the use of untreated controls limits the interpretation of the study. Blinding was not possible in this work either, as the control group did not receive any preoperative treatment. Central randomisation was performed, but the process is not clearly described. Therefore, it is not possible to ascertain whether allocation concealment was maintained or not. Additionally, random assignment was stratified by age, gender, weight loss and tumour length. As also pointed out by the authors of the Cochrane review, it is unclear whether an intention-to-treat (ITT) analysis had been performed, as information on withdrawals was missing or unclear.

As the validity of the report is not sufficient, the described effect cannot be considered as clinically relevant.

MRC Allum et al[17] described the long-term results of a previously published study by the same group in 2002. If information was not found in the last studies, we checked if the needed information was available in the first publication[28]. Taking this into consideration, 11 of the 33 validity criteria were not met (33.3%) by MRC Allum et al[17]. Four items were not applicable. As in the previous study, the use of untreated controls limits the interpretation of the study. Blinding was also not possible because the control group did not receive any preoperative treatment. Due to the use of the minimisation method, allocation concealment is not maintained. A power calculation is missing.

In the previous publication by the same group in 2002[28], the sponsor appointed the writing committee, which interpreted data, wrote the report and submitted it for publication. The risk profiles of the two groups are slightly different with a certain probability of unbalanced risk distribution in favour of the intervention group regarding age and degree of dysphagia.

As the validity of the report is not sufficient, the described effect cannot be considered as clinically relevant.

Meta-analyses

Figure 2 shows the result of the meta-analysis when the three analysed studies were excluded. A total of seven studies were included. One study (Law et al[19]) showed a positive and statistically significant result in favour of the use of preoperative CTx before resection of thoracic esophageal cancer. Six of the included studies were not statistically significant by themselves. The new meta-analysis estimate had a HR of 0.94 with a 95%CI (0.81; 1.09) under assumption of a fixed-effect model and a HR of 0.92 with a 95%CI (0.75; 1.13) under assumption of a mixed-effect model. Regardless of the assumed model, the new estimate does not confirm the advantage of preoperative CTx for resectable thoracic esophageal cancer. The estimate of the original meta-analysis was 0.88 with a 95%CI (0.80; 0.96). The exclusion of the three studies completely changed the result of the meta-analysis. In Boonstra et al[16], only 24.2% of the items on the CONSORT checklist were inappropriately answered, so we assumed that the validity of this study was enough to be included in the meta-analysis. We performed a new meta-analysis excluding only Ychou et al[15] and MRC Allum et al[17] (Figure 3). We found a HR of 0.90 with a 95%CI (0.81; 1.00) under assumption of a fixed-effect model and a HR of 0.90 with a 95%CI (0.78; 1.05) under assumption of a mixed-effect model. Again, regardless of the assumed model, the new estimate does not confirm the advantage of preoperative CTx for resectable thoracic esophageal cancer.

Figure 2
Figure 2 Meta-analysis of seven studies after excluding the three analysed studies. HR: Hazard ratio; N(T): Number of patients in the experimental group; N(C): Number of patients in the control group; W(fixed): Weight assigned to the study by using a fixed-effect model; W(random): Weight assigned to the study by using a random-effect model.
Figure 3
Figure 3 Meta-analysis of eight studies after excluding the studies by Ychou and MRC Allum. HR: Hazard ratio; N(T): Number of patients in the experimental group; N(C): Number of patients in the control group; W(fixed): Weight assigned to the study by using a fixed-effect model; W(random): Weight assigned to the study by using a random-effect model.

Finally, we performed a second meta-analysis (Figure 4) also excluding Law et al[19], which found a positive and statistically significant result as well. After the exclusion of all four studies with positive and statistically significant results, the new meta-analysis consisted of only six statistically non-significant studies. The new meta-analysis estimate was HR 1.04 with a 95%CI (0.88; 1.22), confirming the lack of a survival advantage for patients undergoing preoperative CTx before resection of the thoracic esophageal cancer.

Figure 4
Figure 4 Meta-analysis of six studies after excluding all studies which found a statistically significant survival advantage in the experimental group. HR: hazard ratio; N(T): Number of patients in the experimental group; N(C): Number of patients in the control group; W(fixed): Weight assigned to the study by using a fixed-effect model; W(random): Weight assigned to the study by using a random-effect model.
DISCUSSION

In the present manuscript, we assessed the validity of three studies included in the meta-analysis by Kidane et al[18], which supports the results of improved survival in patients treated with preoperative CTx for resectable thoracic esophageal cancer. It is important to identify possible bias in the three studies which support the result of the meta-analysis because bias jeopardizes validity. We demonstrated that these three studies are not valid enough to be included in a Cochrane review. When excluded from the meta-analysis, the overall result of the meta-analysis is no longer significant.

We will first illustrate the problems we discovered in the three mathematically most influential studies supporting the conclusions and, in a second step, discuss our findings after performing the new meta-analyses.

Common problems in all studies

The lack of a placebo-controlled and blinded study affects the validity of the three studies and, consequently, the validity of the review. As discussed in our previous work[26], without a placebo control, it is impossible to differentiate between specific pharmacological and placebo effects. A placebo effect is defined as the “…response of a subject to a substance or any procedure known to be without specific therapeutic effect for the condition being treated[29].” Several studies demonstrated that perceptual characteristics of drugs[30], the route of administration[31], laboratory tests[32], diagnosis[33] and the doctor-patient relationship play an important role in the outcome of an illness[34-37]. Information regarding treatment or no treatment alone is sufficient to elicit a placebo effect[38]. Moreover, patients’ and doctors’ preferences could also have influenced the results in an open study[39]. Patients assigned to the control group feel disadvantaged because they expect to be treated. Furthermore, when there is no concealment of treatment allocation, the randomisation procedure is compromised because of conscious or subconscious bias[40]. It is important to perform an ITT analysis to maintain the balance distribution of risk factors between groups achieved by a randomisation procedure. A correct ITT analysis was only conducted in the studies by MRC Allum et al[17] and Ychou et al[15]. These aspects collectively affect the validity of the reports and, therefore, the described effects cannot be considered as clinically relevant.

Specific problems of the study by Ychou et al[15]

In the study by Ychou et al[15], a minimisation method is used. Minimisation[41-44], a type of dynamic allocation, is gaining popularity especially in clinical cancer trials. In this design, the new subject’s treatment assignment is determined by evaluating the potential covariate imbalance that would result if he or she were assigned to the treatment or to the control group[45]. Minimisation aims at achieving balance over a large number of prespecified prognostic factors simultaneously. We raise concerns over this design, as it compromises adequate generation of an allocation sequence and concealment in this study. Investigators using minimisation can actually determine the group to which a prospective subject would be allocated and then decide whether this is positive or negative in terms of creating an imbalance in some key predictor of outcome not considered in the imbalance function. Despite adding randomisation, so that the treatment that minimises the imbalance function for a given patient is not necessarily allocated to that patient, there is a high probability of this being the case[46]. The European Medicines Agency’s Committee[47] states that “dynamic allocation is strongly discouraged”.

Specific problems of the study by Boonstra et al[16]

In this study, as in the study by Ychou et al[15], the randomisation process is not exhaustively described; they only mentioned that a central randomisation took place. A description of the randomisation process is completely lacking. Aside from this problem, which is extremely relevant, we find that this study was conducted well in comparison to the other two.

Specific problems of the study by MRC Allum et al[17]

This study reports long follow-up results of a previously published study by the same authors (2002)[28]. As in the study by Ychou et al[15], minimisation was used, raising the same concerns as previously described. A power calculation is completely missing. Finally, a sponsor-related conflict of interest was identified by our analysis.

As recently shown by Shnier et al[48], financial conflicts of interest and relationships between guideline authors and drug companies are common and represent a source of bias in studies. As authoritative value is assigned to guidelines, it is important to develop formal policies to limit the potential influence of any conflict of interest on guideline recommendations. This is the only way to improve the quality of medical publications. Only valid studies are reliable studies. For an expert pool aiming to publish guidelines, it is necessary to scrutinise the validity of single studies and of meta-analyses as well, as low-quality studies can lead to a distortion of the summary-effect estimate[49].

In the second part of our analysis we performed the meta-analysis first without the three analysed studies and showed that the result of the meta-analysis is no longer significant. This result coincides with previous big studies and the original meta-analysis by Malthaner et al[13]. Moreover, as we find that the study by Boonstra et al[16] was quite well done in comparison to the other two, we performed a new meta-analysis excluding only the studies of Ychou et al[15] and MRC Allum et al[17]. The estimate also showed no benefit of preoperative CTx before surgical resection. As expected, when all studies with positive results are eliminated from the meta-analysis, the estimate is not significant.

Implications for practice

According to the results of the Cochrane review, preoperative CTx should be used for patients with resectable thoracic esophageal cancer. However, it is important to note that some of the included trials contain limitations so that definitive assessments of this topic should be delayed until future trials are properly developed. The three analysed studies that were chosen because of their attributed weights are not sufficiently valid to be included in a meta-analysis, which is also true for most of the other studies included.

Despite finding several inconsistencies and substantial deficits in the included high-power studies, the aim of this work is not primarily to identify the best therapeutic treatment for esophageal cancer, but to increase awareness of the quality of studies and their impact on medical treatment when used in meta-analyses or Cochrane reviews. Especially studies that were performed before implementation of the CONSORT checklist show a variety of inconsistencies that would exclude publication according to current quality standards. High-quality RCTs decrease the risk of inherent bias and therefore receive higher attributed weight in meta-analyses. The inclusion of several low-power studies with serious deficits can overpower well conducted studies and change the outcome.

The analysed Cochrane review was published in 2015; only three included studies were performed after 2009, but seven before 2001, some even dating back to before the 1990s. At that time, no standardised reporting procedure, like the CONSORT checklist, existed. Therefore, the findings are quite heterogeneous. The three most powerful studies were the last ones published and still show a substantial lack in standardisation according to the CONSORT checklist, which was first published in 1996 and revised in 2001 and 2010.

As the incidence of esophageal carcinoma is relatively low, studies often include adenocarinoma and squamous-cell carcinoma without discrimination. Even worse, in some of the studies adjuvant treatment was not only CTx, but sometimes also RCTx for squamous-cell carcinoma. Both inherently different carcinoma types with different neoadjuvant treatment regimens were included in a single group. To analyse the role of neoadjuvant CTx in this context, two groups needed to be established: RTx alone vs RCTx as neoadjuvant therapy as performed by Herskovic et al[50]. In this paper, adenocarcinoma and squamous-cell carcinoma of the esophagus were also put into one group.

Multimodale therapy in patients with esophageal cancer is now the standard treatment in most centres today and is recommended in several national guidelines[51-52].

In Germany, S3 guidelines for esophageal carcinoma were updated in 2018[51]. Several newer publications, usually multicentric randomised controlled studies, were taken into account.

The evaluated Cochrane review by Kidane is not mentioned in the current German S3 guideline for the standardised treatment of esophageal carcinoma. However, the analysed studies by Ychou et al[15], Boonstra et al[16] and MRC Allum et al[17] with observed inconsistencies are mentioned and included. Thanks to the authors of the German S3 guideline, the current data is critically presented and not all study results are included in the recommendation for standardised treatment: “In squamous cell carcinoma, no consistent increase in survival after CTx alone – despite the positive study by Boonstra – could be observed by metaanalyses.” (page 101 German S3 guidelines AWMF-Registernummer: 021/023OL).

In conclusion, multimodal therapy of advanced esophageal carcinoma represents the current gold standard for treatment. We observed several deficits of the analysed studies in the Cochrane review by Kidane. Interestingly, this review was not taken into account in the current german S3 guideline for treatment of esophageal carcinoma, and the analyzed single studies are there critically reviewed and set in context with similar research papers. Well performed (multicentric) randomized controlled studies are needed to be analysed together in a meta-analyse. High-quality single studies are required, as they determine the outcome of meta-analyses that can influence the recommendations of national guidelines.

Footnotes

Manuscript source: Invited manuscript

Specialty type: Oncology

Country of origin: Germany

Peer-review report classification

Grade A (Excellent): 0

Grade B (Very good): B

Grade C (Good): 0

Grade D (Fair): 0

Grade E (Poor): 0

P-Reviewer: Okamoto H S-Editor: Zhang L L-Editor: A E-Editor: Liu MY

References
1.  Ferlay J, Soerjomataram I, Ervik M, Dikshit R, Eser S, Mathers C, Rebelo M, Parkin DM, Forman D, Bray F. Globocan 2012 v1.0, Cancer Incidence and Mortality Worldwide: IARC CancerBase No. 11 [Internet] Lyon, France: International Agency for Research on Cancer 2013. pp. cited 2015-06-25.  Available from: http://globocan.iarc.fr..  [PubMed]  [DOI]  [Cited in This Article: ]
2.  Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68:394-424.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 53206]  [Cited by in F6Publishing: 51531]  [Article Influence: 8588.5]  [Reference Citation Analysis (122)]
3.  DeMeester TR, Barlow AP. Surgery and current management for cancer of the esopha-gus. Current problems in Surgery. 1988;535-605 doi:10.1016/0011-3840(88)90027-5.  [PubMed]  [DOI]  [Cited in This Article: ]
4.  Lerut T. Esophageal surgery at the end of the millennium. Journal of Thoracic and Cardiovascular Surgery. 1998;116:1-20.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 36]  [Cited by in F6Publishing: 35]  [Article Influence: 1.3]  [Reference Citation Analysis (0)]
5.  DeVita Jr, Vincent T, Rosenberg, Steven A, Lawrence, Theodore S, DeVita VT, Lawrence TS, Rosenberg SA. Cancer of the Esophagus. DeVita, Hellman, and Rosenberg‘s Cancer: Principles and Practice of Oncology. Philadelphia: Wolters Kluwer; 2015; 574-612.  [PubMed]  [DOI]  [Cited in This Article: ]
6.  M Fok, J Wong, SWK Cheng and SYK Law. A comparison of outcome after resection for squa-mous cell carcinomas and adenocarcinomas of the esophagus and cardia. Surgery Gy-necology and Obstetrics. 1992;175:107-112.  [PubMed]  [DOI]  [Cited in This Article: ]
7.  Lerut T, De Leyn P, Coosemans W, Van Raemdonck D, Scheys I, LeSaffre E. Surgical strategies in esophageal carcinoma with emphasis on radical lymphadenectomy. Ann Surg. 1992;216:583-590.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 235]  [Cited by in F6Publishing: 246]  [Article Influence: 7.7]  [Reference Citation Analysis (0)]
8.  Liebermann MD, Shriver CD, Bleckner S, Burt M. Carcinoma of the esophagus. Prog-nostic significance of histologic type. Journal of Thoracic and Cardiovascular Surgery. 1995;130-138.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 76]  [Cited by in F6Publishing: 80]  [Article Influence: 2.8]  [Reference Citation Analysis (0)]
9.  Orringer MB. Multimodality therapy for esophageal carcinoma--update. Chest. 1993;103:406S-409S.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 21]  [Cited by in F6Publishing: 21]  [Article Influence: 0.7]  [Reference Citation Analysis (0)]
10.  Keditsu KK, Jiwnani S, Karimundackal G, Pramesh CS. Multimodality management of esophageal cancer. Indian J Surg Oncol. 2013;4:96-104.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 12]  [Cited by in F6Publishing: 14]  [Article Influence: 1.3]  [Reference Citation Analysis (0)]
11.  Garg PK, Sharma J, Jakhetiya A, Goel A, Gaur MK. Preoperative therapy in locally advanced esophageal cancer. World J Gastroenterol. 2016;22:8750-8759.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in CrossRef: 14]  [Cited by in F6Publishing: 14]  [Article Influence: 1.8]  [Reference Citation Analysis (1)]
12.  Li F, Ding N, Zhao Y, Yuan L, Mao Y. The current optimal multimodality treatments for oesophageal squamous-cell carcinoma: A systematic review and meta-analysis. Int J Surg. 2018;60:88-100.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 4]  [Cited by in F6Publishing: 13]  [Article Influence: 2.2]  [Reference Citation Analysis (0)]
13.  Malthaner R, Fenlon D. Preoperative CTx for resectable thoracic esophageal cancer. Cochrane Database of Systematic Reviews. 2001;1 [DOI 10.1002/14651858.CD001556.pub2].  [PubMed]  [DOI]  [Cited in This Article: ]
14.  Urschel JD, Vasan H, Belwett CJ. A meta-analysis of randomized controlled trials that compared neoadjuvant CTx and surgery to surgery alone for resectable esophageal cancer. American Journal of Surgery. 2002;183:274-9.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 170]  [Cited by in F6Publishing: 184]  [Article Influence: 8.4]  [Reference Citation Analysis (0)]
15.  Ychou M, Boige V, Pignon JP, Conroy T, Bouché O, Lebreton G, Ducourtieux M, Bedenne L, Fabre JM, Saint-Aubert B, Genève J, Lasser P, Rougier P. Perioperative chemotherapy compared with surgery alone for resectable gastroesophageal adenocarcinoma: an FNCLCC and FFCD multicenter phase III trial. J Clin Oncol. 2011;29:1715-1721.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1216]  [Cited by in F6Publishing: 1378]  [Article Influence: 106.0]  [Reference Citation Analysis (0)]
16.  Boonstra JJ, Kok TC, Wijnhoven BP, van Heijl M, van Berge Henegouwen MI, Ten Kate FJ, Siersema PD, Dinjens WN, van Lanschot JJ, Tilanus HW, van der Gaast A. Chemotherapy followed by surgery versus surgery alone in patients with resectable oesophageal squamous cell carcinoma: long-term results of a randomized controlled trial. BMC Cancer. 2011;11:181.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 114]  [Cited by in F6Publishing: 125]  [Article Influence: 9.6]  [Reference Citation Analysis (0)]
17.  Allum WH, Stenning SP, Bancewicz J, Clark PI, Langley RE. Long-term results of a randomized trial of surgery with or without preoperative chemotherapy in esophageal cancer. J Clin Oncol. 2009;27:5062-5067.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 662]  [Cited by in F6Publishing: 718]  [Article Influence: 47.9]  [Reference Citation Analysis (0)]
18.  Kidane B, Coughlin S, Vogt K, Malthaner R. Preoperative chemotherapy for resectable thoracic esophageal cancer. Cochrane Database Syst Rev. 2015;CD001556.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 30]  [Cited by in F6Publishing: 44]  [Article Influence: 4.9]  [Reference Citation Analysis (0)]
19.  Law S, Fok M, Chow S, Chu KM, Wong J. Preoperative CTx versus surgical therapy alone for squamous cell carcinoma of the esophagus: a prospective randomized trial. Journal of Thoracic and Cardiovascular Surgery. 1997;114:210-217.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 286]  [Cited by in F6Publishing: 291]  [Article Influence: 10.8]  [Reference Citation Analysis (0)]
20.  Ancona E, Ruol A, Santi S, Merigliano S, Sileni VC, Koussis H, Zaninotto G, Bonavina L, Peracchia A. Only pathologic complete responce to neoadjuvant chemotheraphy improves significantly the long term survival of patients with resectable esophageal squamous cell carcinoma: Final report of a randomized, controlled trial of preoperative chemotherapy versus surgery alone. Cancer. 2001;91:2165-2174.  [PubMed]  [DOI]  [Cited in This Article: ]
21.  Kelsen DP, Ginsberg R, Pajak TF, Sheahan DG, Gunderson L, Mortimer J, Estes N, Haller DG, Ajani J, Kocha W, Minsky BD, Roth JA. Chemotherapy followed by surgery compared with surgery alone for localized esophageal cancer. N Engl J Med. 1998;339:1979-1984.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1025]  [Cited by in F6Publishing: 939]  [Article Influence: 36.1]  [Reference Citation Analysis (0)]
22.  Maipang T, Vasinanukorn P, Petpichetchian C, Chamroonkul S, Geater A, Chansawwaang S, Kuapanich R, Panjapiyakul C, Watanaarepornchai S, Punperk S. Induction chemotherapy in the treatment of patients with carcinoma of the esophagus. J Surg Oncol. 1994;56:191-197.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 112]  [Cited by in F6Publishing: 116]  [Article Influence: 3.9]  [Reference Citation Analysis (0)]
23.  Nygaard K, Hagen S, Hansen HS, Hatlevoll R, Hultborn R, Jakobsen A, Mäntyla M, Modig H, Munck-Wikland E, Rosengren B. Pre-operative radiotherapy prolongs survival in operable esophageal carcinoma: a randomized, multicenter study of pre-operative radiotherapy and chemotherapy. The second Scandinavian trial in esophageal cancer. World J Surg. 1992;16:1104-9; discussion 1110.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 459]  [Cited by in F6Publishing: 422]  [Article Influence: 13.2]  [Reference Citation Analysis (0)]
24.  Roth JA, Pass HI, Flanagan MM, Graeber GM, Rosenberg JC, Steinberg S. Randomized clinical trial of preoperative and postoperative adjuvant chemotherapy with cisplatin, vindesine, and bleomycin for carcinoma of the esophagus. Journal of Thoracic Cardiovascular Surgery. 1988;96:242-8.  [PubMed]  [DOI]  [Cited in This Article: ]
25.  Schlag PM. Randomized trial of preoperative chemotherapy for squamous cell cancer of the esophagus. The Chirurgische Arbeitsgemeinschaft Fuer Onkologie der Deutschen Gesellschaft Fuer Chirurgie Study Group. Arch Surg. 1992;127:1446-1450.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 214]  [Cited by in F6Publishing: 195]  [Article Influence: 6.7]  [Reference Citation Analysis (0)]
26.  Manzini G, Henne-Bruns D, Kremer M. Validity of studies suggesting postsurgical chemotherapy for resectable gastric cancer: critical appraisal of randomised trials. BMJ Open Gastroenterol. 2017;4:e000138.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 6]  [Cited by in F6Publishing: 6]  [Article Influence: 0.9]  [Reference Citation Analysis (0)]
27.  Moher D, Schulz KF, Altman DG; CONSORT GROUP (Consolidated Standards of Reporting Trials). The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med. 2001;134:657-662.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 815]  [Cited by in F6Publishing: 799]  [Article Influence: 34.7]  [Reference Citation Analysis (0)]
28.  Medical Research Council Esophageal Cancer working Party. Surgical resection with or without preoperative chemotheraphy in esophageal cancer: a randomized controlled trial. The Lancet. 2002;359:1727-1733.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1121]  [Cited by in F6Publishing: 1056]  [Article Influence: 48.0]  [Reference Citation Analysis (0)]
29.  Benedetti F, Amanzio M. The neurobiology of placebo analgesia: from endogenous opiois to cholecystokinin. Progr Neurobiol. 1997;52:109-125.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 164]  [Cited by in F6Publishing: 168]  [Article Influence: 6.2]  [Reference Citation Analysis (0)]
30.  Buckalew LW, Coffield KE. An investigation of drug expectancy as a function of capsule color and size and preparation form. J Clin Psychopharmacol. 1982;2:245-248.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 87]  [Cited by in F6Publishing: 88]  [Article Influence: 2.1]  [Reference Citation Analysis (0)]
31.  Kowalczyk M, Wall A, Turek T, Kulej M, Scigała K, Kawecki J. Computerized tomography evaluation of cortical bone properties in the tibia. Ortop Traumatol Rehabil. 2007;9:187-197.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 3]  [Cited by in F6Publishing: 3]  [Article Influence: 0.2]  [Reference Citation Analysis (0)]
32.  Sox HC, Margulies I, Sox CH. Psychologically mediated effects of diagnostic tests. Ann Intern Med. 1981;95:680-685.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 131]  [Cited by in F6Publishing: 140]  [Article Influence: 3.3]  [Reference Citation Analysis (0)]
33.  Thomas KB. General practice consultations: is there any point in being positive? Br Med J (Clin Res Ed). 1987;294:1200-1202.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 281]  [Cited by in F6Publishing: 248]  [Article Influence: 6.7]  [Reference Citation Analysis (0)]
34.  Bass MJ, Buck C, Turner L, Dickie G, Pratt G, Robinson HC. The physician‘s actions and the outcome of illness in family practice. J Fam Pract. 1986;23:43-47.  [PubMed]  [DOI]  [Cited in This Article: ]
35.  Gracely RH, Dubner R, Deeter WR, Wolskee PJ. Clinician‘s expectations influence placebo analgesia. Lancet. 1985;1:1-43.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 214]  [Cited by in F6Publishing: 177]  [Article Influence: 4.5]  [Reference Citation Analysis (0)]
36.  Greenfield S, Kaplan S, Ware JE. Expanding patient involvement in care. Effects on patient outcomes. Ann Intern Med. 1985;102:520-528.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 980]  [Cited by in F6Publishing: 926]  [Article Influence: 23.7]  [Reference Citation Analysis (0)]
37.  Stewart MA. Effective physician-patient communication and health outcomes: a review. CMAJ. 1995;152:1423-1433.  [PubMed]  [DOI]  [Cited in This Article: ]
38.  Waber RL, Shiv B, Carmon Z, Ariely D. Commercial features of placebo and therapeutic efficacy. JAMA. 2008;299:1016-1017.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 257]  [Cited by in F6Publishing: 256]  [Article Influence: 16.0]  [Reference Citation Analysis (0)]
39.  Porzsolt F, Eisemann M, Habs M, Wyer P. Form follows function: pragmatic controlled trials (PCTs) have to answer different questions and require different designs than randomized controlled trials (RCTs). Z Gesundh Wiss. 2013;21:307-313.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 17]  [Cited by in F6Publishing: 16]  [Article Influence: 1.3]  [Reference Citation Analysis (0)]
40.  Altman DG, Schulz KF. Statistics notes: Concealing treatment allocation in randomised trials. BMJ. 2001;323:446-447.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 142]  [Cited by in F6Publishing: 132]  [Article Influence: 5.7]  [Reference Citation Analysis (0)]
41.  Pocock SJ, Simon R. Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics. 1975;31:103-115.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1628]  [Cited by in F6Publishing: 1547]  [Article Influence: 31.6]  [Reference Citation Analysis (0)]
42.  Taves DR. Minimization: a new method of assigning patients to treatment and control groups. Clin Pharmacol Ther. 1974;15:443-453.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 612]  [Cited by in F6Publishing: 545]  [Article Influence: 10.9]  [Reference Citation Analysis (0)]
43.  Wei L-J. A class of designs for sequential clinical trials. Journal of the American Statistical Association. 1977;72:382-386.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 81]  [Cited by in F6Publishing: 35]  [Article Influence: 0.7]  [Reference Citation Analysis (0)]
44.  Wei L-J. The adaptive biased coin design for sequential experiments. The Annals of Statistics. 1978;6:92-100.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 132]  [Cited by in F6Publishing: 134]  [Article Influence: 2.9]  [Reference Citation Analysis (0)]
45.  Xu Z, Proschan M, Lee S. Validity and power considerations on hypothesis testing under minimization. Stat Med. 2016;35:2315-2327.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 12]  [Cited by in F6Publishing: 12]  [Article Influence: 1.5]  [Reference Citation Analysis (0)]
46.  Berger VW. Minimization, by its nature, precludes allocation concealment, and invites selection bias. Contemp Clin Trials. 2010;31:406.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 36]  [Cited by in F6Publishing: 36]  [Article Influence: 2.6]  [Reference Citation Analysis (0)]
47.  Committee for Proprietary Medicinal Products (CPMP). Committee for Proprietary Medicinal Products (CPMP): points to consider on adjustment for baseline covariates. Stat Med. 2004;23:701-709.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 89]  [Cited by in F6Publishing: 101]  [Article Influence: 5.1]  [Reference Citation Analysis (0)]
48.  Shnier A, Lexchin J, Romero M, Brown K. Reporting of financial conflicts of interest in clinical practice guidelines: a case study analysis of guidelines from the Canadian Medical Association Infobase. BMC Health Serv Res. 2016;16:383.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 20]  [Cited by in F6Publishing: 23]  [Article Influence: 2.9]  [Reference Citation Analysis (0)]
49.  Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol. 2010;25:603-605.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8858]  [Cited by in F6Publishing: 10867]  [Article Influence: 776.2]  [Reference Citation Analysis (0)]
50.  Herskovic A, Martz K, al-Sarraf M, Leichman L, Brindle J, Vaitkevicius V, Cooper J, Byhardt R, Davis L, Emami B. Combined chemotherapy and radiotherapy compared with radiotherapy alone in patients with cancer of the esophagus. N Engl J Med. 1992;326:1593-1598.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1514]  [Cited by in F6Publishing: 1401]  [Article Influence: 43.8]  [Reference Citation Analysis (0)]
51.  S3-Leitlinie Diagnostik und Therapie der Plattenepithelkarzinome und Adenokarzinome des Esophagus.  Publiziert bei: AWMF online. Leitlinien programm Onkoligie 2018. Version 2.0..  [PubMed]  [DOI]  [Cited in This Article: ]
52.  Martin-Richard M, Díaz Beveridge R, Arrazubi V, Alsina M, Galan Guzmán M, Custodio AB, Gómez C, Muñoz FL, Pazo R, Rivera F. SEOM Clinical Guideline for the diagnosis and treatment of esophageal cancer (2016). Clin Transl Oncol. 2016;18:1179-1186.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 14]  [Article Influence: 1.8]  [Reference Citation Analysis (0)]