P- Reviewer: Ni Y S- Editor: Qiu S L- Editor: A E- Editor: Liu SQ
Published online Mar 26, 2016. doi: 10.5662/wjm.v6.i1.101
Peer-review started: August 10, 2015
First decision: November 3, 2015
Revised: December 17, 2015
Accepted: December 29, 2015
Article in press: January 4, 2016
Published online: March 26, 2016
Randomized controlled trials (RCTs) are the gold standard in terms of study design, however, in the surgical setting conducting RCTs can often be unethical or logistically impossible. Case-control studies should become the major study design used in surgical research when RCTs are unable to be conducted and definitely replacing case series which offer little insight into surgical outcomes and disease processes.
Core tip: Case-control studies should be utilized more often in the surgical setting for research purposes. They offer many advantages to other study designs, especially when the option of conducting a randomized clinical trial may be impractical or not ethically feasible.
Citation: Cao AM, Cox MR, Eslick GD. Study design in evidence-based surgery: What is the role of case-control studies? World J Methodol 2016; 6(1): 101-104
The hierarchy of study design is well ingrained in determining the quality and subsequent acceptance of clinical evidence (Figure 1). Randomised controlled trials (RCT) are considered the gold standard study design and the “most scientifically rigorous method for hypothesis testing”, with results from many non-randomised trials prejudiced by doubts of study reliability, bias and accuracy[1-3]. Yet in certain aspects of surgery, RCTs may be difficult to conduct and indeed the number of surgical RCTs is known to be limited in comparison.
RCTs involve the comparison of outcomes after random allocation of a particular intervention to a patient group with a control group whilst case-control studies (CCS) involve observing outcome differences between patients with a particular disease (cases) and those without the disease (control). It is commonly accepted that results from RCTs provide superior evidence in the evaluation of a therapeutic intervention when compared to CCS. However, there are many considerations that result in flaws in this concept especially in surgery. Difficulties in standardising surgical technique, variable learning curves in introducing new or modifications of an operation and difficulties in recruiting patients leading to underpowered studies need to be recognised. In fact results from poorly designed RCTs can have the undue advantage of being perceived and accepted as the “superior study design” with more robust findings. The aim of this paper is to explore various factors influencing the role of CCS in the surgical context and provide recommendations to improving the quality of CCS.
The strength of CCS lie in its ability to recruit larger sample sizes, resultant increase in the power of studies, lower cost and the ability to be conducted in “greater timeliness” (Table 1). CCS also have the ability to report rare infrequent adverse effects, e.g., bile duct injuries in laparoscopic cholecystectomies[8,9]. As CCS may be performed by researchers with limited resources, larger patient populations are able to be recruited compared with RCTs which generally require more expert support from epidemiologists and require financial support. Lack of funding and resource constraints have been cited as major obstacles in conducting RCTs. Inadequate sample sizes lead to underpowered RCTs which may miss clinically important benefits and lead to type II error. Type II error is the failure to reject the null hypothesis when it is false, i.e., False negative results.
|Advantages of case control studies||Disadvantages of case control studies|
|Ability to investigate low incidence outcomes||Risk of bias|
|Ability to recruit large sample size||Confounding factors|
|Relative ease and efficiency||Requires careful selection of controls|
|May be conducted in shorter time frame||Weaker evidence of causality (20)|
|Relatively low cost||Blinding is not possible|
A particular strength of CCS is the inclusion of data from practical clinical scenarios. RCTs, whilst limiting potential confounding variables, provide evidence from data collected from highly rigid experimental models. In investigating certain surgical techniques such as laparoscopic cholecystectomy, strict criteria such as those excluding obese patients and patients with multiple comorbidities are likely to lead to results inconsistent with the clinical setting and limit the practicality of findings. In patients who have rare or life threatening illnesses, it will be difficult to include them in RCT. In addition unlike CCS, RCT also tend to limit the spectrum of disease represented compared to observational studies[2,6].
Non-randomised observational studies such as CCS and cohort studies are more prone to bias than RCT due to lack of randomisation. The randomisation process aims to minimise systematic error and eliminate or at least equilibrate confounding factors between both treatment and control groups. It is more difficult for observational studies to allow for this equilibration and hence is more prone to bias. Without randomisation, it may be unclear why certain patients were assigned to a particular intervention whilst others were not. However, whilst randomisation can limit bias, it may not be feasible or ethical in the surgical context. For example, it may be unethical to deny one group of patients the treatment benefits of well established “gold standard” interventions. In addition, it may be difficult to recruit patients who will leave their choice of treatment up to chance alone and accept the process of randomisation.
Whilst it is more difficult for CCS to account for confounding factors, it is not impossible without randomisation. Matching controls with cases is one potential method. Matching where controls are specifically selected for their similarity to the treatment group in particular characteristics such as age, sex, socioeconomic status, body mass index, etc., can be used to equilibrate potential confounders in CCS.
Furthermore in surgery, allocation concealment and blinding may be impractical and unethical. In most major surgical procedures, it would be unethical to expose patients in the control groups to the risks of sham operations. Whilst various techniques have been used in the blinding of patients in surgery including the use of multiple wound dressings over intact skin, the efficacy of such blinding techniques is unclear.
The concern that observational studies can bias evidence by finding stronger treatment associations than RCTs has been reported in the literature[7,16]. However comparisons between results for observational and RCTs in other studies have shown results to be similar between the two in most outcomes. For example one study analysed the results of meta-analyses comparing RCTs and well-designed observational studies (cohort and case control studies) on a range of treatments including hypertension treatment and CHD, Bacillus Calmette-Guerin vaccine in tuberculosis, mammography screening for breast cancer and found results from observational studies “did not systematically overestimate the magnitude of exposure-outcome associations reported in RCTs”[2,7]. An explanation for the noted differences in some studies between RCT and CCS potentially results from less robustly designed CCS were used to generate generalised conclusions regarding observational studies.
It would be imprudent to argue that CCS provide a superior level of evidence to RCT. However, CCS can often provide additional and more clinically relevant evidence that can complement data derived from RCTs. There are various means of ensuring high quality CCS. Recommendations to ensuring sound CCS evidence include: (1) encourage use of STROBE statement to ensure adequate reporting of outcomes; (2) develop an exhaustive database of baseline characteristics and variables during data collection stage of CCS; (3) design CCS to test the clinical applicability and generalizability of results from RCT rather than formulating hypothesis to investigate[17-19]; (4) ensure adequate statistical power and sample size by performing sample size and power calculations prior to the initiation of studies; (5) appropriate statistical techniques for the clinical question, e.g., Propensity analysis to match patients, use of risk adjusted statistical models; and (6) encourage sound methodology techniques such as intention to treat and adequate follow-up.
Well-designed RCTs undoubtedly provide powerful estimates of treatment effects. However, they are time-consuming, costly, difficult to conduct especially in surgery and can be misinterpreted when data is extrapolated outside the experiment sample. CCS on the other hand have the ability to recruit large sample sizes, are more efficient to conduct and allow for the examination of variables in the clinical setting. It is unfortunate that CCS are often undervalued and under-utilised in surgery. RCT and CCS provide evidence that is complementary to each other. Greater understanding is required in appraising RCT and CCS in the surgical environment.
|1.||DiPietro NA. Methods in epidemiology: observational study designs. Pharmacotherapy. 2010;30:973-984. [PubMed] [DOI]|
|2.||Concato J. Observational versus experimental studies: what’s the evidence for a hierarchy? NeuroRx. 2004;1:341-347. [PubMed] [DOI]|
|3.||Hillner BE, Mandelblatt J. Caring for older women with breast cancer: can observational research fill the clinical trial gap? J Natl Cancer Inst. 2006;98:660-661. [PubMed] [DOI]|
|4.||Ahmed Ali U, van der Sluis PC, Issa Y, Habaga IA, Gooszen HG, Flum DR, Algra A, Besselink MG. Trends in worldwide volume and methodological quality of surgical randomized controlled trials. Ann Surg. 2013;258:199-207. [PubMed] [DOI]|
|5.||Curry JI, Reeves B, Stringer MD. Randomized controlled trials in pediatric surgery: could we do better? J Pediatr Surg. 2003;38:556-559. [PubMed] [DOI]|
|6.||McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D. Randomised trials in surgery: problems and possible solutions. BMJ. 2002;324:1448-1451. [PubMed] [DOI]|
|7.||Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med. 2000;342:1878-1886. [PubMed] [DOI]|
|8.||Cao AM, Eslick GD, Cox MR. Early laparoscopic cholecystectomy is superior to delayed acute cholecystitis: a meta-analysis of case-control studies. Surg Endosc. 2016;30:1172-1182. [PubMed] [DOI]|
|9.||MacMahon S, Collins R. Reliable assessment of the effects of treatment on mortality and major morbidity, II: observational studies. Lancet. 2001;357:455-462. [PubMed] [DOI]|
|10.||Comber H, Perry IJ. Observational studies for intervention assessment. Lancet. 2001;357:2141-2142. [PubMed] [DOI]|
|11.||Gattellari M, Ward JE, Solomon MJ. Randomized, controlled trials in surgery: perceived barriers and attitudes of Australian colorectal surgeons. Dis Colon Rectum. 2001;44:1413-1420. [PubMed] [DOI]|
|12.||Maggard MA, O’Connell JB, Liu JH, Etzioni DA, Ko CY. Sample size calculations in surgery: are they done correctly? Surgery. 2003;134:275-279. [PubMed] [DOI]|
|13.||Jahn A, Razum O. Observational studies for intervention assessment. Lancet. 2001;357:2141. [PubMed] [DOI]|
|14.||McLeod RS. Issues in surgical randomized controlled trials. World J Surg. 1999;23:1210-1214. [PubMed] [DOI]|
|15.||de Graaf MA, Jager KJ, Zoccali C, Dekker FW. Matching, an appealing method to avoid confounding? Nephron Clin Pract. 2011;118:c315-c318. [PubMed] [DOI]|
|16.||Chalmers TC, Celano P, Sacks HS, Smith H. Bias in treatment assignment in controlled clinical trials. N Engl J Med. 1983;309:1358-1361. [PubMed] [DOI]|
|17.||von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61:344-349. [PubMed] [DOI]|
|18.||Hannan EL. Randomized clinical trials and observational studies: guidelines for assessing respective strengths and limitations. JACC Cardiovasc Interv. 2008;1:211-217. [PubMed] [DOI]|