Opinion Review
Copyright ©The Author(s) 2020. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastroenterol. Jun 7, 2020; 26(21): 2682-2690
Published online Jun 7, 2020. doi: 10.3748/wjg.v26.i21.2682
Cost of postoperative complications: How to avoid calculation errors
Roberto De la Plaza Llamas, José M Ramia
Roberto De la Plaza Llamas, José M Ramia, Department of General and Digestive Surgery, Hospital Universitario de Guadalajara, Guadalajara 19002, Spain
ORCID number: Roberto De la Plaza Llamas (0000-0001-9501-5480); José M Ramia (0000-0003-1186-953X).
Author contributions: De la Plaza Llamas R and Ramia JM conceived the study and drafted the manuscript; De la Plaza Llamas R design the study and performed the research; Both authors approved the final version of the article.
Conflict-of-interest statement: The authors have no conflict of interest to declare.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Corresponding author: Roberto De la Plaza Llamas, FACS, MD, MSc, PhD, Associate Professor, Doctor, Surgeon, Department of General and Digestive Surgery, Hospital Universitario de Guadalajara, Calle Donante de Sangre s/n, Guadalajara 19002, Spain. rdplazal@sescam.jccm.es
Received: December 27, 2019
Peer-review started: December 27, 2019
First decision: March 27, 2020
Revised: March 27, 2020
Accepted: May 20, 2020
Article in press: May 20, 2020
Published online: June 7, 2020

Abstract

Postoperative complications (PC) are a basic health outcome, but no surgery service in the world records and/or audits the PC associated with all the surgical procedures it performs. Most studies that have assessed the cost of PC suffer from poor quality and a lack of transparency and consistency. The payment system in place often rewards the volume of services provided rather than the quality of patients’ clinical outcomes. Without a thorough registration of PC, the economic costs involved cannot be determined. An accurate, reliable appraisal would help identify areas for investment in order to reduce the incidence of PC, improve surgical results, and bring down the economic costs. This article describes how to quantify and classify PC using the Clavien-Dindo classification and the comprehensive complication index, discusses the perspectives from which economic evaluations are performed and the minimum postoperative follow-up established, and makes various recommendations. The availability of accurate and impartially audited data on PC will help reduce their incidence and bring down costs. Patients, the health authorities, and society as a whole are sure to benefit.

Key Words: Morbidity, Postoperative complications, Costs and cost analysis, Economic evaluation, Healthcare costs, Opportunity cost, Clavien-Dindo classification, Comprehensive complication index

Core tip: No surgery service in the world registers and/or audits the postoperative complications (PC) of all the surgical procedures it performs. Most economic studies of PC are of poor quality; without an accurate registration of PC, their costs cannot be reliably determined. The article describes ways of quantifying and classifying PC, discusses the perspectives from which their economic evaluation and monitoring can be approached, and makes recommendations. An accurate assessment of PC and their costs will allow us to determine which areas need investment in order to lower the incidence of PC, improve surgical results, and reduce economic costs.



INTRODUCTION

Expenditure on health as a proportion of gross domestic product in OECD countries in 2017 ranged from 17.2% in the United States to 4.2% in Turkey[1]. Because of the rising economic costs (EC) deriving from the growing use of expensive technology and population aging, and the need to invest in other basic health areas or global issues such as combating the effects of climate change, expenditure on health must achieve maximum efficiency, incorporating other health benefits and improving the quality of all.

Postoperative complications (PC) are a basic health outcome. Their presence has a negative effect on patients’ quality of life in the short and/or long term and increases the consumption of human, technical and economic resources. An accurate appraisal of the effects of PC would allow us to determine which specific areas are need of investment in order to reduce their incidence, improve surgical results, raise patients’ quality of life, and bring down EC.

CURRENT SITUATION

At present no surgery service anywhere in the world systematically records all the morbidity associated with the surgical procedures it performs[2] or submits records of this kind to objective and impartial audit[3]. What is more, the analyses of the EC of PC lack consistency and transparency in terms of the presentation of reports on costs and resource use[4]. Without a complete and impartial registry of PC, the associated EC cannot be determined.

A prospective study by our group evaluating the PC of all surgical procedures at a surgery service found a morbidity rate of 27.7%. Depending on the complexity of the surgery, the PC ranged from 10.7% to 71.4%. In addition, among patients with complications, 51.5% presented two events or more[5].

Even patient death has a negative economic effect. One cost analysis of 1200 patients undergoing major surgery showed that hospital costs incurred by a death were significantly higher than the costs of Clavien-Dindo classification (CDC) grade I to IIIb complications (although they were significantly lower than those associated with grade IV complications in all surgeries)[6]. In pancreatic surgery the costs in patients rescued from PC were 64% lower than those incurred in patients who died due to the complication[7]. In studies validating the association of scales with EC, deceased patients should not be included in the comparative statistical analysis due to possible distortions caused by non-survivor bias[5,6]. In addition to EC, the loss of patient productivity should be taken into account.

Obviously, PC must be monitored after discharge. For example, surgical site infections are a frequent PC[7]; an incidence of 15% has been reported[8,9] and of these as many as 58% occur after hospital discharge[9].

From the hospital perspective, the minimum follow-up period of PC and their EC must be at least 90 d and readmissions must be considered. One prospective study of all interventions carried out at a general surgery service found an overall readmission rate after 90 d of 5.5%[10].

Patel et al[4] conducted a systematic literature review to assess the relationship between EC and the severity of complications in 38 studies that included pancreatic, hepatic, gynecological, urological and thoracic surgeries, and also analyzed the methodologies used. They concluded that EC depend on the type and severity of the complication and are due mainly to prolonged hospitalization. Many of the studies included used general or national databases rather than databases from the hospitals themselves, and all 38 studies included only payer/hospital (i.e., not societal) costs. It was also observed that the studies did not tend to use classifications of complication severity[4].

The association between PC severity and increased EC has been demonstrated in major abdominal surgery[6,11-14], and in another study of all procedures performed at a surgery service[15]. In the latter study for example, even the presence of mild CDC grade 1 complications was found to double postoperative costs from the hospital perspective after cholecystectomy[15].

In some cases it has been observed that complications are profitable for hospitals[16,17]. This is because the payment system rewards the volume of services provided to patients, rather than the quality of the patient’s clinical outcome[18].

ISSUES TO CONSIDER IN THE ANALYSIS OF EC ASSOCIATED WITH PC

To determine the EC related to PC, we must: (1) Quantify and register all PC and the patients affected; (2) Identify and quantify all postoperative EC; and (3) Assess the difference in costs between complicated and non-complicated patients.

Let us now look at these three points in more detail.

Quantification, registration and classification of all PC

The first step is to determine what we mean by PC. We agree with the definition of the CDC, which considers a complication to be any negative event occurring during hospitalization[5,19-21], even if it is asymptomatic or only remotely related to the initial surgical procedure[5,15]. This classification is intended to avoid subjectivity and to ensure that all observers apply the same criteria. This fact entails a series of important consequences: (1) Postoperative patients who have had complications may be classified as uncomplicated: (A) In this case PC-related EC may be attributed to the mean or median costs of patients without complications. This overestimates the costs of patients without complications and brings down the mean EC of patients with complications included in the non-complicated group; and (B) Patients with complications who are wrongly classified as non-complicated do not appear in the PC group (these complications are usually minor). This misclassification pushes up the mean or median costs estimated for particular procedures associated with PC, and erroneously underestimates the total cost of patients with complications; and (2) This leads to errors of classification of diagnosis-related groups (DRG). DRG classify patients who have similar clinical characteristics and similar treatment costs. The purpose of their use is to relate a hospital’s case load to the demand for resources and the costs incurred. Therefore, if a surgery service has a more complex case load according to DRG (as is the case when complications are recorded), this means that the hospital is treating patients who need more hospital resources and, by extension, represent a higher economic outlay. Failure to record all PC would have grave economic consequences for the management unit responsible for patient care because it would be assigned fewer resources (i.e., the amount corresponding to non-complex DRG).

Secondly, PC must be recorded thoroughly. To achieve this, physicians must be formally obliged to prospectively document all complications affecting the patient during or after surgery. But this does not usually occur. In order to record all PC, researchers must evaluate the medical and (above all) the nursing logs and, if possible, specific complications forms in the electronic medical record[5]. An external audit is mandatory: The health authorities have an essential role to play here.

Thirdly, PC must be classified according to a severity or complexity system. We recommend the CDC, published in 2004[19]. This classification system is the most used worldwide, and the article in which it was described currently has 12318 citations[22]. The problem with the CDC is that it categorizes the entire postoperative period according to the most serious complication. This means that patients with two or more adverse events are underrepresented, and it has been demonstrated that between 44 and 51.5% of patients with morbidity at general surgery services have two or more complications[5,23]. To overcome this problem, in 2013 Slankamenac et al[24] developed a score that takes into account all the PC classified according to the CDC and summarizes them on a scale between 0 (without complications) and 100 (patient death). This scale, called the comprehensive complication index (CCI), has been validated both clinically[5] and from an economic perspective[14,15].

It would be reasonable to include intraoperative complications in the calculation of EC. In a systematic review of abdominal surgery, Garbens et al[25] observed that intraoperative adverse events significantly increased the total EC, although the studies were of poor quality. However, despite the fact that a clinical classification of these complications has been published[26], these events can only be reported by the surgical team; the surgeons may be fearful of the damage to their reputation of an audit (or its legal consequences) and may be tempted to conceal the complication. This means that there may be major differences and biases in the evaluation of clinical and economic results between different surgery services. In addition, even if they are not taken into account, most of the intraoperative complications that affect EC would be detected during the postoperative period. In any case sensitivity studies might shed further light on this matter.

Identification and quantification of all postoperative EC

The economic evaluation of PC may be performed from a variety of perspectives, depending on who is responsible for paying for them and the intended use of the results of the analysis. Different perspectives will consider different costs and time periods. Examples of these perspectives are the patient, the institution (e.g., the hospital), the target group for specific services (e.g., rehabilitation), the public health service, the public sector, or the sum of all these perspectives (i.e., the social perspective)[27].

ECs are not distributed equally across the hospital stay. Taheri et al[28] concluded that approximately 40% of variable costs are incurred during the first three days of admission. A reduction in length of stay of up to one full day at the end of admission reduces the total cost of care on average by 3% or less. Obviously, the occurrence of PC would change this trend.

It seems reasonable to assess EC until a specific time after patient discharge. Frequently, part of the costs of the care is transferred to the outpatient setting, in particular in patients who are discharged early. So, how long after the operation should EC be assessed? Ideally, all PC should be considered until the patient recovers from the consequences: Imagine a patient with a brain injury that leads to a definite tetraplegia. So, in the best-case scenario, all EC should be considered throughout the patient's life, although this may be impossible to achieve.

As noted by Drummond et al[27] in other economic evaluations, the follow-up time depends on the perspective applied. From the hospital's perspective, we think that the time period should be a minimum of 90 d, or preferably when the patient is able to return to work or their normal activities. However, in the case of certain PC such as a reconstruction of the intestine after an anastomosis dehiscence, this period could be lengthened. If the calculation is made from the perspective of other health agencies, the follow-up period might be for example one year, while from the perspective of the patient and/or family it might be considerably longer, even the entire lifetime[27]. So, the study should indicate the follow-up time used and why.

Table 1 summarizes the different EC included in the literature which we believe should be taken into account in the economic evaluation of PC.

Table 1 Types of costs to consider in the economic evaluation of postoperative complications.
Direct costs
1 Health-related
1.1 Postoperative resources used/consumed by the patient during hospitalization:
(1) Shared or overhead costs. Financing of administrative infrastructure and fixed installations. Resources used by different services: Administration, records, electricity, heating, laundry, etc.
(2) Central services: Documentation, computer services, …
(3) Installations and catering.
(4) Health and non-health staff: Nurses, doctors, ancillary staff from the surgery service ...
(5) Use of other surgical and medical services at the hospital.
(6) Diagnostic procedures: Laboratory, radiological and endoscopic examinations, pathology studies, …
(7) Pharmacy.
(8) Surgical, endoscopic and radiological procedures. Ideally, recording of consumables for each procedure.
(9) Dressing materials.
1.2 Future postoperative costs related to the surgery performed and the postoperative complications arising:
(1) Primary care and hospital visits.
(2) Rehabilitation.
(3) Diagnostic procedures: Analytical, radiological and endoscopic examinations, ...
(4) Pharmacy.
(5) Readmissions associated with the surgery, however remote the relation.
(6) Care centres (for example, rehabilitation, convalecence or other long-term care centres)
(7) Health transport: Ambulances.
(8) Prostheses, wheelchaires, …
2 Non-health-related
2.1 Monetary:
(1) Social services: Home care, notification systems, …
(2) Time taken to receive medical care.
(3) Transport costs incurred by patients and families.
(4) Care for patient’s dependents.
(5) Adaptation of the home, vehicles, …
2.2 Non-monetary (opportunity cost):
(1) Time taken by the patient, families and friends in care of the illness.
Indirect costs
1.1 Productivity lost by patients (those in employment):
(1) Temporary or definitive incapacity for work.
(2) Readaptation in the workplace.
(3) Death.
1.2 Productivity lost by family members (those in employment):
(1) Paid time off work.
(2) Non-paid time off work.
1.3 Associated with the legal issues that may occur.

Given that most evaluations of PC-related EC have been carried out from the perspective of the hospital[4], we would like to make some recommendations. It has been argued that taking micro costs into account (i.e., each component of the resources used per patient)[27] is expensive. We do not share this view; with the electronic medical record, the investment for determining these costs is minimal but these costs are still assigned equally, independently of the individual cost. Some examples are: (1) The pharmacy costs associated with each patient are recorded electronically at almost all hospitals; (2) The cost of consumables used during surgical interventions (sutures, gauze, endostaplers, meshes) is very easy to calculate using a barcode reader and to incorporate into the medical record. In advanced surgery this EC is very high. However, operating room EC are usually assessed according to the time taken, regardless of the intervention, and not according to the material consumed; and (3) The cost of diagnostic tests, if they are assigned a unit cost. Consumables should also be individualized.

Specific measurement of each patient’s use of these resources is essential to distinguish between the costs of patients with and without PC. Not taking them into account creates an obvious bias[15]. In addition, we believe that the practice of adjusted the number of stays by DRG is inappropriate[29].

Since the economic evaluation of PC involves the comparison of two clinical evolutions, any costs that are common to both do not need to be quantified; we are interested only in differential costs and results, rather than totals. Any preoperative EC, including the EC of the initial or index surgery or PC caused by postoperative cancer treatments, should be excluded from the analysis.

To determine the cost of PC related to a specific procedure, the operations compared should be as similar as possible in terms of complexity and the patients compared should also be similar in terms of severity. The use of the Charlson morbidity scale can be particularly useful[30].

Table 2 describes the recommendations to be followed in the assessment of EC associated with PC and the data we think should be described in the economic study.

Table 2 Recommendations for the economic evaluations of costs associated with postoperative complications of surgical procedures.
No.Data to be described in the economic study
1Type of study
2Center: Particular characteristics
3Period of inclusion of cases
4Population and subgroups analyzed; clarify why they were chosen
5Surgical interventions considered
6Methods for managing the heterogeneity of the population and the surgical techniques used; clarify the possible influence of this heterogeneity on the results
7If possible, describe the characteristics of the severity or complexity of patients (ASA, Charlson, Frailty ...)
8Patients excluded and why
9Patients and procedures lost for analysis. Economic data lost. Clarify whether these can compromise the validity of the results
10Definition of complication
11How complications are classified (better, Clavien-Dindo classification and comprehensive complication index)
12Sources used to obtain data on the complications (specific forms, medical and nursing evolution comments)
13Perspective from which the economic evaluation is carried out: the patient, the specific institution (e.g., the hospital), the target group for specific services e.g., rehabilitation), the Public Health Service, the Public Sector in general or from all perspectives (social perspective)
14Health-related costs (direct and indirect), non-health-related costs (table 1). Sources used to obtain data. Specify the costs obtained en bloc (for example, general costs) those obtained in microcosts (radiology, pharmacy, ...) In the estimation of operating room costs, specify whether the cost of the consumable material used in each patient is individualized
15Describe the postoperative follow-up time in which complications and costs will be evaluated for each perspective and procedure. From the hospital perspective, consider 90 d minimum
16Methods for converting costs into a common monetary base and the exchange rate. If it includes more than one calendar year, specify corrections made for inflation
17Biases and limitations of the study, and measures used to reduce them
18Conflict of interests of researchers. Linking researchers to the surgical service and hospital
19Source of study financing
Assessment of the differences in costs between complicated and non-complicated patients

Finally, to determine the EC of PC, the postoperative EC of patients with PC should be subtracted the postoperative EC of patients without PC. That is, costs in patients with and without complications should be compared. If the PC are classified according to the CDC and/or the CCI we can calculate the costs associated with each grade of the CDC or score on the CCI in any patient[15].

DISCUSSION

We believe that studies of EC associated with PC should ideally take into account the costs described in Table 1 and the recommendations and clarifications in Table 2. Researchers should specify the ones they use. Some principles have been previously published in the CHEERS statement[31].

In the best-case scenario, it would be possible to compare PC and their associated EC in different surgical procedures and in different hospitals at regional or global level. This would indicate the specific problems of each service and procedure, allow the introduction of improvements, and determine quality and efficiency benchmark services. But for this to be possible, care must be taken to measure the same phenomenon with the same tools: That is, assessing the complexity of the procedure and the severity of the patient, the concept of complication, the classification system, the evaluation perspective, the level of the hospital, and so on (Table 2).

Once the PC associated with a surgical procedure at a particular service have been identified, the introduction of a protocol aimed to allow early diagnosis and treatment of complications can help bring down morbidity and mortality rates and can also prove cost-effective[11].

Obviously, without access to results expressed as real and audited PC of the various surgical interventions, it is impossible to consider the centralization of complex surgical procedures or the accreditation of surgical units or of training courses[3]. When these results are available, efficiency studies can be performed.

The training of surgeons is essential for reducing PC. Higher costs have been associated with surgeons who have performed fewer interventions[32,33] and a higher number of intraoperative complications[32].

Outside the public health structure, the payment system in place may reward the volume of services provided to the patient rather than the patient’s clinical outcome. Despite its obvious flaws, many hospitals have used this system for a long time[18]. Unfortunately, today there are no economic or social incentives to reduce PC, and the reporting of the results of surgery services all over the world continues to lack transparency. In fact, it is astonishing that in the year 2019 so little should be known about a basic quality outcome of surgical services such as PC. It is an insult to science, patients and society as a whole.

We believe that PC at all surgery services should be audited in an objective and impartial manner by an external assessor. The health authorities would be able to carry out this process quickly and cheaply; using the electronic medical record, the assessment can be carried out in an average of 5 to 10 min per patient[2]. The audit should be maintained over a period of time and should be performed in all patients, not random samples. The audit of the results of the services is also likely to improve surgeons' practices due to the Hawthorne effect, though the mechanisms of this effect and its magnitudes need to be elucidated[34].

The audit can enumerate the complications associated with each surgical procedure and measure their severity, thus identifying the areas where investment in interventions and improvements is most necessary to raise quality and reduce EC. Patients must be informed of the possible outcomes of the operation they are to undergo. The results available at present are unreliable, since they are based on evaluations made by surgeons at the service. This practice underestimates the number of PC, for many reasons: the fact that the better the recording system, the worse the results; the surgeon’s sensation of personal failure; the fear of comparison with other surgeons and/or services; the fear that complex processes will be centralized, and so on. The responsibility for the fact that this audit of all surgical procedures has not been carried out so far in any surgery service in the world lies with the surgeons and the health authorities. Thorough and accurate data on the occurrence of PC will help to bring down their incidence and obviously the EC as well. In addition, it is a mandatory measure to ensure transparency.

CONCLUSION

First of all, all the PC that are associated with a particular surgical procedure must be objectively determined. This is the responsibility of the surgical community and forms part of its commitment towards its patients, society, and science as a whole. Simply recording all PC is likely to reduce their incidence, improve quality and significantly lower EC. The information obtained would help to identify the surgical techniques and clinical management practices with the best postoperative results, to implement preventive measures in order to reduce PC and select the benchmarking services that should take charge of training. Currently, surgical training is not led by the services with the best clinical results. The EC saved can be reinvested in health and/or social areas. Complications should be audited externally and impartially by the health authorities, and in a universal, permanent manner in all surgical patients. Hospitals with the best results should be rewarded economically and not according to a payment system that may favour poor outcomes. This payment would only be justified in centres of proven quality.

ECs associated with PC should also be reported in a standardized way[31]. Appropriate and accurate methods should be used to track the use of resources and to estimate EC. Accounting must be accurate and detailed. Although the most frequently used perspective is that of the hospital, health authorities and health managers should bear in mind that EC associated with PC are much higher than the published figures, if the estimation includes future direct EC, non-health-related EC and indirect EC which have major repercussions for society and for patients.

Specifically, we think that recording and auditing of PC is likely will become the measure of the greatest impact on healthcare transparency, quality of care and the reduction of ECs in the coming decades.

Footnotes

Manuscript source: Invited manuscript

Specialty type: Gastroenterology and hepatology

Country/Territory of origin: Spain

Peer-review report’s scientific quality classification

Grade A (Excellent): A

Grade B (Very good): B, B, B

Grade C (Good): 0

Grade D (Fair): 0

Grade E (Poor): 0

P-Reviewer: Chiu KW, Kayaalp C, Niu ZS, Slomiany BL S-Editor: Dou Y L-Editor: A E-Editor: Ma YJ

References
1.  Health expenditure as a percentage of gross domestic product in selected countries in 2018. Statista.  Available from: https://www.statista.com/statistics/268826/health-expenditure-as-gdp-percentage-in-oecd-countries/.  [PubMed]  [DOI]
2.  De la Plaza Llamas R, Ramia JM. Postoperative complications in gastrointestinal surgery: A "hidden" basic quality indicator. World J Gastroenterol. 2019;25:2833-2838.  [PubMed]  [DOI]
3.  de la Plaza Llamas R, García Gil JM, Ramia Ángel JM. Standardizing the treatment of esophagogastric junction tumors: Centralization, registries and surgical training. Cir Esp. 2019;97:609-611.  [PubMed]  [DOI]
4.  Patel AS, Bergman A, Moore BW, Haglund U. The economic burden of complications occurring in major surgical procedures: a systematic review. Appl Health Econ Health Policy. 2013;11:577-592.  [PubMed]  [DOI]
5.  de la Plaza Llamas R, Ramia Ángel JM, Bellón JM, Arteaga Peralta V, García Amador C, López Marcano AJ, Medina Velasco AA, González Sierra B, Manuel Vázquez A. Clinical Validation of the Comprehensive Complication Index as a Measure of Postoperative Morbidity at a Surgical Department: A Prospective Study. Ann Surg. 2018;268:838-844.  [PubMed]  [DOI]
6.  Vonlanthen R, Slankamenac K, Breitenstein S, Puhan MA, Muller MK, Hahnloser D, Hauri D, Graf R, Clavien PA. The impact of complications on costs of major surgical procedures: a cost analysis of 1200 patients. Ann Surg. 2011;254:907-913.  [PubMed]  [DOI]
7.  Sullivan E, Gupta A, Cook CH. Cost and Consequences of Surgical Site Infections: A Call to Arms. Surg Infect (Larchmt). 2017;18:451-454.  [PubMed]  [DOI]
8.  Delgado-Rodríguez M, Gómez-Ortega A, Sillero-Arenas M, Llorca J. Epidemiology of surgical-site infections diagnosed after hospital discharge: a prospective cohort study. Infect Control Hosp Epidemiol. 2001;22:24-30.  [PubMed]  [DOI]
9.  Daneman N, Lu H, Redelmeier DA. Discharge after discharge: predicting surgical site infections after patients leave hospital. J Hosp Infect. 2010;75:188-194.  [PubMed]  [DOI]
10.  de la Plaza Llamas R. Validación y aplicabilidad clínica del Comprehensive Complication Index en una población de pacientes intervenidos en un servicio de cirugía general y del aparato digestivo. Estudio prospectivo. Thesis, University of Alcalá. 2018.  Available from: https://www.educacion.gob.es/teseo/mostrarRef.do?ref=444441.  [PubMed]  [DOI]
11.  Straatman J, Cuesta MA, de Lange-de Klerk ES, van der Peet DL. Hospital cost-analysis of complications after major abdominal surgery. Dig Surg. 2015;32:150-156.  [PubMed]  [DOI]
12.  Gani F, Johnston FM, Nelson-Williams H, Cerullo M, Dillhoff ME, Schmidt CR, Pawlik TM. Hospital Volume and the Costs Associated with Surgery for Pancreatic Cancer. J Gastrointest Surg. 2017;21:1411-1419.  [PubMed]  [DOI]
13.  Goense L, van Dijk WA, Govaert JA, van Rossum PS, Ruurda JP, van Hillegersberg R. Hospital costs of complications after esophagectomy for cancer. Eur J Surg Oncol. 2017;43:696-702.  [PubMed]  [DOI]
14.  Staiger RD, Cimino M, Javed A, Biondo S, Fondevila C, Périnel J, Aragão AC, Torzilli G, Wolfgang C, Adham M, Pinto-Marques H, Dutkowski P, Puhan MA, Clavien PA. The Comprehensive Complication Index (CCI®) is a Novel Cost Assessment Tool for Surgical Procedures. Ann Surg. 2018;268:784-791.  [PubMed]  [DOI]
15.  de la Plaza Llamas R, Hidalgo Vega Á, Latorre Fragua RA, López Marcano AJ, Medina Velasco AA, Díaz Candelas DA, García Gil JM, Ramia Ángel JM. The Cost of Postoperative Complications and Economic Validation of the Comprehensive Complication Index: Prospective Study. Ann Surg. 2019;.  [PubMed]  [DOI]
16.  Krupka DC, Sandberg WS, Weeks WB. The impact on hospitals of reducing surgical complications suggests many will need shared savings programs with payers. Health Aff (Millwood). 2012;31:2571-2578.  [PubMed]  [DOI]
17.  Eappen S, Lane BH, Rosenberg B, Lipsitz SA, Sadoff D, Matheson D, Berry WR, Lester M, Gawande AA. Relationship between occurrence of surgical complications and hospital finances. JAMA. 2013;309:1599-1606.  [PubMed]  [DOI]
18.  Reinhardt UE. Making surgical complications pay. JAMA. 2013;309:1634-1635.  [PubMed]  [DOI]
19.  Dindo D, Demartines N, Clavien PA. Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg. 2004;240:205-213.  [PubMed]  [DOI]
20.  Clavien PA, Sanabria JR, Strasberg SM. Proposed classification of complications of surgery with examples of utility in cholecystectomy. Surgery. 1992;111:518-526.  [PubMed]  [DOI]
21.  Clavien PA, Barkun J, de Oliveira ML, Vauthey JN, Dindo D, Schulick RD, de Santibañes E, Pekolj J, Slankamenac K, Bassi C, Graf R, Vonlanthen R, Padbury R, Cameron JL, Makuuchi M. The Clavien-Dindo classification of surgical complications: five-year experience. Ann Surg. 2009;250:187-196.  [PubMed]  [DOI]
22.  Classification of Surgical Complications Overview of attention for article published in Annals of Surgery, August 2004. Wolters Kluwer, Artic. Metrics [cited 2019 Dec 27].  Available from: https://wolterskluwer.altmetric.com/details/4800046/citations.  [PubMed]  [DOI]
23.  Clavien PA, Vetter D, Staiger RD, Slankamenac K, Mehra T, Graf R, Puhan MA. The Comprehensive Complication Index (CCI®): Added Value and Clinical Perspectives 3 Years "Down the Line". Ann Surg. 2017;265:1045-1050.  [PubMed]  [DOI]
24.  Slankamenac K, Graf R, Barkun J, Puhan MA, Clavien PA. The comprehensive complication index: a novel continuous scale to measure surgical morbidity. Ann Surg. 2013;258:1-7.  [PubMed]  [DOI]
25.  Garbens A, Goldenberg M, Wallis CJD, Tricco A, Grantcharov TP. The cost of intraoperative adverse events in abdominal and pelvic surgery: A systematic review. Am J Surg. 2018;215:163-170.  [PubMed]  [DOI]
26.  Rosenthal R, Hoffmann H, Clavien PA, Bucher HC, Dell-Kuster S. Definition and Classification of Intraoperative Complications (CLASSIC): Delphi Study and Pilot Evaluation. World J Surg. 2015;39:1663-1671.  [PubMed]  [DOI]
27.  Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW.   Methods for the economic evaluation of health care programmes. 4th ed. Oxford: Oxford University Press; 2015.  [PubMed]  [DOI]
28.  Taheri PA, Butz DA, Greenfield LJ. Length of stay has minimal impact on the cost of hospital admission. J Am Coll Surg. 2000;191:123-130.  [PubMed]  [DOI]
29.  de la Plaza Llamas R, Gemio Del Rey IA, Ramia JM. Discrepancies in the analysis of frequency, type of complications and costs of outlying patients in general and digestive surgery. Cir Esp. 2019;97:420-421.  [PubMed]  [DOI]
30.  Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373-383.  [PubMed]  [DOI]
31.  Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, Augustovski F, Briggs AH, Mauskopf J, Loder E; CHEERS Task Force. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Value Health. 2013;16:e1-e5.  [PubMed]  [DOI]
32.  Wallenstein MR, Ananth CV, Kim JH, Burke WM, Hershman DL, Lewin SN, Neugut AI, Lu YS, Herzog TJ, Wright JD. Effect of surgical volume on outcomes for laparoscopic hysterectomy for benign indications. Obstet Gynecol. 2012;119:709-716.  [PubMed]  [DOI]
33.  Wright JD, Hershman DL, Burke WM, Lu YS, Neugut AI, Lewin SN, Herzog TJ. Influence of surgical volume on outcome for laparoscopic hysterectomy for endometrial cancer. Ann Surg Oncol. 2012;19:948-958.  [PubMed]  [DOI]
34.  McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol. 2014;67:267-277.  [PubMed]  [DOI]