Retrospective Cohort Study Open Access
Copyright ©The Author(s) 2019. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastroenterol. Dec 7, 2019; 25(45): 6668-6680
Published online Dec 7, 2019. doi: 10.3748/wjg.v25.i45.6668
Prognostic value of risk scoring systems for cirrhotic patients with variceal bleeding
Xin-Xing Tantai, Na Liu, Long-Bao Yang, Zhong-Cao Wei, Cai-Lan Xiao, Ya-Hua Song, Jin-Hai Wang
Xin-Xing Tantai, Na Liu, Long-Bao Yang, Zhong-Cao Wei, Cai-Lan Xiao, Ya-Hua Song, Jin-Hai Wang, Division of Gastroenterology, The Second Affiliated Hospital, Xi’an Jiaotong University, Xi’an 710004, Shaanxi Province, China
ORCID number: Xin-Xing Tantai (0000-0002-4105-6890); Na Liu (0000-0003-3246-9674); Long-Bao Yang (0000-0001-7731-0910); Zhong-Cao Wei (0000-0001-8931-680X); Cai-Lan Xiao (0000-0001-8476-5008); Ya-Hua Song (0000-0002-5988-8599); Jin-Hai Wang (0000-0003-1365-4727).
Author contributions: Tantai XX, Wang JH and Liu N contributed to study concept and design; Xiao CL and Song YH contributed to data collection; Yang LB and Wei ZC contributed to data verification; Tantai XX, Wang JH and Liu N contributed to data management, interpretation and supervision; Tantai XX contributed to statistical analysis and drafted the manuscript; Tantai XX, Wang JH and Liu N contributed to critical revision of the manuscript.
Institutional review board statement: This study was reviewed and approved by the Ethics Committee of The Second Affiliated Hospital of Xi’an Jiaotong University.
Informed consent statement: Patients were not required to give informed consent to the study because the analysis used anonymous clinical data that were obtained after each patient agreed to treatment by written consent.
Conflict-of-interest statement: All authors declare no conflicts of interest related to this article.
Data sharing statement: No additional data are available.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Corresponding author: Jin-Hai Wang, MD, Professor, Division of Gastroenterology, The Second Affiliated Hospital, Xi’an Jiaotong University, No. 157, Xiwu Road, Xi’an 710004, Shaanxi Province, China. jinhaiwang@hotmail.com
Telephone: +86-29-87679335 Fax: +86-29-87679368
Received: September 26, 2019
Peer-review started: September 26, 2019
First decision: November 4, 2019
Revised: November 13, 2019
Accepted: November 16, 2019
Article in press: November 16, 2019
Published online: December 7, 2019

Abstract
BACKGROUND

Acute variceal bleeding is one of the deadliest complications of cirrhosis, with a high risk of in-hospital rebleeding and mortality. Some risk scoring systems to predict clinical outcomes in patients with upper gastrointestinal bleeding have been developed. However, for cirrhotic patients with variceal bleeding, data regarding the predictive value of these prognostic scores in predicting in-hospital outcomes are limited and controversial.

AIM

To validate and compare the overall performance of selected prognostic scoring systems for predicting in-hospital outcomes in cirrhotic patients with variceal bleeding.

METHODS

From March 2017 to June 2019, cirrhotic patients with acute variceal bleeding were retrospectively enrolled at the Second Affiliated Hospital of Xi’an Jiaotong University. The clinical Rockall score (CRS), AIMS65 score (AIMS65), Glasgow-Blatchford score (GBS), modified GBS (mGBS), Canada-United Kingdom-Australia score (CANUKA), Child-Turcotte-Pugh score (CTP), model for end-stage liver disease (MELD) and MELD-Na were calculated. The overall performance of these prognostic scoring systems was evaluated.

RESULTS

A total of 330 cirrhotic patients with variceal bleeding were enrolled; the rates of in-hospital rebleeding and mortality were 20.3% and 10.6%, respectively. For in-hospital rebleeding, the discriminative ability of the CTP and CRS were clinically acceptable, with area under the receiver operating characteristic curves (AUROCs) of 0.717 (0.648-0.787) and 0.716 (0.638-0.793), respectively. The other tested scoring systems had poor discriminative ability (AUROCs < 0.7). For in-hospital mortality, the CRS, CTP, AIMS65, MELD-Na and MELD showed excellent discriminative ability (AUROCs > 0.8). The AUROCs of the mGBS, CANUKA and GBS were relatively small, but clinically acceptable (AUROCs > 0.7). Furthermore, the calibration of all scoring systems was good for either in-hospital rebleeding or death.

CONCLUSION

For cirrhotic patients with variceal bleeding, in-hospital rebleeding and mortality rates remain high. The CTP and CRS can be used clinically to predict in-hospital rebleeding. The performances of the CRS, CTP, AIMS65, MELD-Na and MELD are excellent at predicting in-hospital mortality.

Key Words: Cirrhosis, Variceal bleeding, Rebleeding, Mortality, Risk score

Core tip: Acute variceal bleeding is one of the most serious complications of cirrhotic patients with a high risk of in-hospital rebleeding and mortality. This study validated and compared the overall performance of eight prognostic scores for predicting in-hospital adverse outcomes in cirrhotic patients with variceal bleeding. We screened out some useful prognostic scores for predicting in-hospital adverse outcomes, especially for predicting in-hospital mortality. These prognostic scores can be easily used for early identification of high-risk patients. For high-risk patients, a transfer to a better hospital, close monitoring and aggressive treatments can help to reduce the risk of in-hospital adverse outcomes.



INTRODUCTION

Cirrhosis is an end-stage liver disease with high mortality and manifests as various degrees of portal hypertension and hepatic dysfunction. Based on the presence or absence of decompensation events (ascites, variceal bleeding, encephalopathy, and jaundice), cirrhosis can be categorized into different prognostic stages: compensated or decompensated cirrhosis[1,2]. Acute variceal bleeding is one of the most life-threatening complications. 22%-61% of cirrhotic patients receiving primary prophylaxis will develop first variceal bleeding during the first two years of follow-up[3]. Furthermore, variceal bleeding is associated with a high risk of rebleeding and mortality. A recent study reported that rebleeding and mortality rates within one month were 25.7% and 15.2%, respectively[4]. Although patient prognosis has improved with modern treatments that can control bleeding, the adverse event rate after variceal bleeding remains high. Therefore, high-risk patients with cirrhosis must be identified early, which can help determine appropriate candidates for risk communication, early intervention, close monitoring, or even early transfer to an intensive care unit.

Some clinical scoring systems have been established and used for predicting clinical outcomes in patients with acute upper gastrointestinal bleeding (UGIB). Among these systems, the most widely used are the clinical Rockall score (CRS), AIMS65 score (AIMS65), and Glasgow-Blatchford score (GBS). These three systems were developed in patients with both nonvariceal UGIB and variceal UGIB and have been widely validated in previous studies for patients presenting with UGIB[5]. However, most of these studies excluded patients with variceal bleeding or included only a small number of these patients. For patients with variceal bleeding, very limited data regarding the prognostic value of these scoring systems are available[5,6]. Furthermore, only a few studies have used these scores to predict the in-hospital outcomes of patients with variceal bleeding, and their conclusions were controversial[4,7-9]. Adverse outcomes during hospitalization are typically the focus of patients and doctors after admission. In addition, two newly created scoring systems have not been externally validated[10,11]. The model for end-stage liver disease (MELD), Child-Turcotte-Pugh score (CTP), and MELD-Na are considered useful for predicting short-term prognoses[12]. The CTP, MELD, MELD-Na, CRS, AIMS65, GBS, modified GBS (mGBS) and Canada-United Kingdom-Australia score (CANUKA) are selected as candidates as they are generally recognized and considered to be useful for predicting short-term outcomes. In addition, validation of these prognostic scores in Chinese patients is rare in terms of predicting in-hospital outcomes. On the other hand, these prognostic scores are easy to calculate using clinical and readily available laboratory variables, so they can be widely used by hospitals of different levels. Therefore, this study aimed to validate and compare the overall performance of these eight prognostic scoring systems for predicting in-hospital outcomes in cirrhotic patients with variceal bleeding.

MATERIALS AND METHODS
Study population

This retrospective cohort study was reported following the TRIPOD statement[13] and conducted at the Second Affiliated Hospital of Xi’an Jiaotong University. Consecutive cirrhotic patients with endoscopically confirmed variceal bleeding between March 2017 and June 2019 were identified by reviewing medical records. The inclusion criteria were adult patients with liver cirrhosis who were admitted to our hospital due to variceal bleeding. The exclusion criteria included the following: (1) Patients who were younger than 18 years old; (2) Patients who refused or could not tolerate endoscopy; (3) Patients with endoscopy-confirmed acute UGIB from non-variceal origins; (4) Transferred patients who were treated at external hospitals; and (5) Patients with incomplete medical records. The diagnosis of liver cirrhosis was made either by clinical assessment with a physical examination, laboratory indices, radiological findings or liver biopsy. Variceal bleeding was diagnosed if gastroscopy showed any of the signs of variceal bleeding according to the Chinese guidelines[14]. All patient management was in line with the standard protocol for cirrhotic patients with acute variceal bleeding[14]. Bleeding patients underwent a preliminary clinical assessment and were resuscitated as soon as possible. Resuscitation measures included endotracheal intubation, oxygen inhalation, multiple peripheral lines or deep vein access, fluid resuscitation, blood transfusions, nasogastric tube insertion, and medication administration (antibiotics, octreotide, somatostatin, terlipressin or anti-hepatic encephalopathy regimens). Balloon tamponade or emergency endoscopic treatment was performed as needed, or the patient was transferred to the intensive care unit. Gastroscopy was scheduled as early as possible, and endoscopic therapies were performed as needed. Transjugular intrahepatic portosystemic stent-shunt (TIPSS) or surgery was considered for cirrhotic patients when endoscopic therapies failed or were unsuitable. These special treatments were performed after obtaining informed consent; if the patients did not consent, they received only medication to control bleeding. This study protocol was approved by the ethics committee of our institution (2019042).

Data collection

The prognostic scores for CTP, MELD, MELD-Na, CRS, AIMS65, GBS, mGBS and CANUKA were calculated for each patient. The formulas and components of the eight scoring systems for calculating the prognostic scores are summarized in Supplementary Table 1. The data required by the scoring systems, demographic data, disease history, laboratory and imaging data were collected within 24 h of hospital admission. Medical record review and data extraction were performed by two trained researchers who were blinded to the study purpose.

Table 1 Baseline characteristics of the included patients.
VariablesValue
No. of patients330
Age, mean ± SD, yr54.9 ± 12.7
Male sex, n (%)203 (61.5)
Etiology, n (%)
Viral229 (69.4)
Alcohol21 (6.4)
Autoimmune34 (10.3)
Other46 (13.9)
Laboratory tests, median (IQR)
White blood cells (109/L)5.4 (3.6-7.8)
Hemoglobin (g/L)80 (66-95)
Platelets (109/L)77 (51-115)
INR1.2 (1.1-1.4)
Total bilirubin (µmol/L)22.6 (16.1-34.8)
ALT (IU/L)23 (15-38)
AST (IU/L)35 (25-56)
Albumin (g/L)31.8 (28.4-35.3)
BUN (mmol/L)8.7 (6.1-12.4)
Creatinine (µmol/L)62.2 (48.5-77.2)
Sodium (mmol/L)138.0 (134.9-140.3)
Vital signs, median (IQR)
SBP (mmHg)110 (98-120)
DBP (mmHg)65 (57-72)
Heart rate (beats/min)88 (76-101)
Location of variceal bleeding, n (%)
Esophageal varices295 (89.4)
Gastric varices35 (10.6)
Grading of esophageal varices
Mild/ moderate/severe, n (%)2 (0.7)/44 (14.9)/249 (84.4)
Types of gastric varices
GOV2/IGV1, n (%)22 (62.9)/13 (37.1)
Ascites, n (%)233 (70.6)
Hepatic encephalopathy, n (%)28 (8.5)
Hepatocellular carcinoma, n (%)73 (22.1)
Bacterial infection, n (%)63 (19.1)
Portal vein thrombosis, n (%)97 (29.4)
Hospital intervention, n (%)
Endoscopic therapy141 (42.7)
TIPSS21 (6.4)
Surgery21 (6.4)
Charlson comorbidity index > 6, n (%)45 (13.6)
Hospital stay (d), median (IQR)13 (9-20)
Scoring system, median (IQR)
CTP7 (6-9)
CTP grade A/B/C, n (%)101 (30.6)/187 (56.7)/42 (12.7)
MELD10 (9-13)
MELD-Na12 (10-16)
CRS2 (1-3)
GBS12 (9-14)
mGBS9 (7-11)
AIMS651 (0-1)
CANUKA10 (8-12)
Study outcomes

Patient follow-up began on the day of admission and ended at patient discharge or death during the same hospitalization period. The primary outcome was in-hospital rebleeding. Secondary outcomes were in-hospital mortality and a composite of in-hospital rebleeding and death. In-hospital rebleeding was defined as recurrence of hematemesis or melena accompanied by hemodynamic instability after the stabilization of vital signs and hemoglobin for at least 24 h. In-hospital mortality was defined as death due to any cause during hospitalization.

Statistical analysis

The sample size estimation was based on the number of positive and negative patients to assess the area under the receiver operating characteristic curve (AUROC). This study hypothesized that the validated scores can effectively predict the risk of in-hospital rebleeding; in other words, the AUROCs of the scores should be greater than 0.5. A previous study reported that the AUROCs of these scores were 0.664-0.756[9]. The minimum value (AUROC = 0.664) was selected as the reference value to obtain the maximum required sample size. The rebleeding rate was reported to be approximately 20% in cirrhotic patients with acute variceal bleeding[4,15]. Using PASS 11.0 software (NCSS, United States), 32 patients with in-hospital rebleeding and 128 nonrebleeding patients were required to achieve 90% power using a one-sided z-test at a significance level of 0.05[16]. Continuous variables with a normal distribution were reported as the mean ± SD and non-normal variables were presented as medians and interquartile ranges (IQRs). Categorical variables were expressed as counts and proportions. The discriminative ability of the prognostic scores was assessed using AUROC with a corresponding 95% confidence interval (CI), and an AUROC greater than 0.7 was considered clinically useful. Comparisons between paired AUROCs were performed using the DeLong test. The optimal threshold in each scoring system was determined by the maximum of the Youden index. The sensitivity, specificity, positive and negative predictive values, and corresponding 95%CIs were calculated for the clinically useful prognostic scores. The calibration of prognostic scores was evaluated by the Hosmer-Lemeshow (H-L) test. A Hosmer-Lemeshow P-value > 0.05 was considered to indicate good calibration. Calibration was also graphically analyzed for prognostic scores with high discriminative ability. Patients were stratified into different risk strata, and then the actual event probability was compared with the predicted event probability within the risk strata. In addition, sensitivity analyses were also performed focusing on patients with esophageal variceal bleeding and patients receiving endoscopic treatments. All data analyses were conducted using SPSS version 22.0 (IBM SPSS Statistics, United States) and MedCalc version 19.0.4 (MedCalc Software bvba, Belgium). A two-tailed P < 0.05 was considered statistically significant.

RESULTS
Patient characteristics

A total of 490 consecutive cirrhotic patients with acute UGIB were screened, and 160 patients were excluded for the following reasons: patients younger than 18 years of age (n = 1), endoscopy was refused or intolerant (n = 42), transferred patients (n = 42), patients with incomplete records (n = 10), and nonvariceal UGIB (n = 65). Finally, 330 independent patients with acute variceal bleeding were included based on the inclusion criteria (Figure 1). The characteristics of the included patients are listed in Table 1. The mean age of these patients was 54.9 ± 12.7 years (range, 25 to 85 years), and 203 patients (61.5%) were male. The vast majority of cirrhosis cases were caused by viral hepatitis; 54.8% patients had HBV, and 13.9% patients had HCV. Alcoholic and autoimmune cirrhosis accounted for 6.4% and 10.3% of the total, respectively. The location of variceal bleeding was esophageal varices in 89.4% of patients and gastric varices in 10.6% of patients. The proportion of severe esophageal varices was 84.4%. Type 1 gastroesophageal varices (GOV1) were classified into esophageal varices. Varices of the stomach fundus included 62.9% type 2 gastroesophageal varices (GOV2) and 37.1% type 1 isolated gastric varices (IGV1). A total of 69.4% of the patients with cirrhosis were categorized as CTP grade B or C. With regard to complications, 70.6% of cirrhotic patients had ascites, 8.5% had hepatic encephalopathy, 22.1% had hepatocellular carcinoma, 19.1% had bacterial infection, and 29.4% had portal vein thrombosis. The Charlson Comorbidity Index (CCI) was used for comorbidity assessment, and the proportion of patients with a CCI greater than 6 points was 13.6%. To control bleeding, 44.8% of patients received only medication, 42.4% received endoscopic treatments, and 12.8% received TIPSS or surgery. The median hospital stay was 13 (9-20) d.

Figure 1
Figure 1 Flowchart of the included patients.
In-hospital rebleeding

Sixty-seven patients underwent in-hospital rebleeding, and the hospital rebleeding rate was 20.3%. The median time interval between admission and rebleeding was 5 d. 40.3% of rebleeding events occurred within 3 d, 70.1% within 7 d, and 29.9% beyond 7 d (Table 2). For predicting in-hospital rebleeding, the AUROCs of the CTP, CRS, MELD-Na, MELD, CANUKA, AIMS65, GBS and mGBS scoring systems were 0.72, 0.72, 0.68, 0.66, 0.66, 0.64, 0.62 and 0.60, respectively (Figure 2A; Table 3). All AUROCs were statistically significant (P < 0.05). Pairwise comparisons of the AUROCs found no significant differences in discriminative ability among the CTP, CRS, MELD-Na, MELD and CANUKA (P > 0.05). Only the AUROCs of the CTP and CRS were clinically acceptable (AUROC > 0.7). Table 4 presents the diagnostic value indices for the clinically useful scoring systems. The cut-off points for the CTP and CRS were 7 and 2, respectively. The sensitivity, specificity, positive predictive value and negative predictive value for the CTP were 74.6%, 63.9%, 34.5% and 90.8%, respectively, and the corresponding values for the CRS were 65.7%, 74.1%, 39.3% and 89.4%, respectively. In addition, the calibration of each scoring system was good, and no significant difference was found between the actual and predicted probabilities (Table 3). A graphical analysis of the scoring system calibration showed a “good” goodness-of-fit for the CTP and CRS (Supplementary Figure 1).

Table 2 Clinical outcomes of the study population, n (%).
OutcomesValue
In-hospital rebleeding67 (20.3)
Admission to rebleeding (time interval, days), median (IQR)5 (3-8)
Rebleeding occurred within 3 d27 (40.3)
Rebleeding occurred within 7 d47 (70.1)
Rebleeding occurred beyond 7 d20 (29.9)
In-hospital mortality35 (10.6)
In-hospital mortality with rebleeding17 (25.4)
In-hospital mortality without rebleeding18 (6.8)
In-hospital adverse outcomes85 (25.8)
Figure 2
Figure 2 Area under the receiver operating curves for the risk scoring systems with regard to in-hospital rebleeding (A), in-hospital mortality (B), and in-hospital adverse outcomes (C). AUROC: Area under the receiver operating characteristic curve; CTP: Child-Turcotte-Pugh score; MELD: Model for end-stage liver disease; CRS: Clinical Rockall score; GBS: Glasgow-Blatchford score; mGBS: Modified Glasgow-Blatchford score; AIMS65: AIMS65 score; CANUKA: Canada-United Kingdom-Australia score.
Table 3 Discrimination and calibration assessments of all the scoring systems for in-hospital adverse outcomes.
ScoreAUROC95%CIP valueAUROC difference95%CIP valueH-L test, P value
In-hospital rebleeding
CTP0.7170.648-0.787< 0.001Reference0.134
CRS0.7160.638-0.793< 0.0010.0016-0.0851-0.08830.97130.062
MELD-Na0.6800.609-0.752< 0.0010.0369-0.0266-0.10000.25470.613
MELD0.6610.588-0.734< 0.0010.0560-0.0026-0.11500.06100.386
CANUKA0.6560.579-0.732< 0.0010.0614-0.0237- 0.14700.15750.186
AIMS650.6420.565-0.719< 0.0010.07530.0120 - 0.13800.01960.321
GBS0.6170.534-0.6990.0030.10100.0153 - 0.18600.02080.041
mGBS0.6000.518-0.6830.0110.11700.0314 - 0.20200.00740.064
In-hospital mortality
CRS0.8830.822-0.943< 0.001Reference0.166
CTP0.8780.825-0.931< 0.0010.0047-0.0620-0.07140.89010.566
AIMS650.8560.787-0.925< 0.0010.0269-0.0390-0.09280.42320.175
MELD-Na0.8560.786-0.925< 0.0010.0271-0.0453-0.09950.46300.636
MELD0.8400.767-0.914< 0.0010.0423-0.0360- 0.12100.29000.472
mGBS0.7340.631-0.838< 0.0010.14800.0552-0.24200.00180.013
CANUKA0.7280.627-0.830< 0.0010.15400.0675-0.24100.00050.046
GBS0.7230.621-0.826< 0.0010.15900.0668-0.25200.00070.004
In-hospital adverse outcomes
CTP0.7840.725-0.843< 0.001Reference0.218
CRS0.7750.709-0.841< 0.0010.0087-0.0650-0.08240.81710.002
MELD-Na0.7390.675-0.803< 0.0010.0446-0.0136-0.10300.13350.723
MELD0.7240.659-0.789< 0.0010.05980.0074-0.11200.02540.464
AIMS650.7110.642-0.780< 0.0010.07280.0160-0.13000.01200.101
CANUKA0.6770.606-0.748< 0.0010.10700.0274-0.18600.00840.023
GBS0.6560.582-0.729< 0.0010.12800.0516-0.20500.00100.008
mGBS0.6490.574-0.723< 0.0010.13500.0586-0.21200.00050.024
Table 4 The predictive value indices for predicting in-hospital adverse outcomes in clinically useful scoring systems.
ScoreYoudenindexCut-offSEN95%CISPE95%CIPPV95%CINPV95%CI
In-hospital rebleeding
CTP0.3851> 774.662.5-84.563.957.8-69.734.529.8-39.490.886.6-93.8
CRS0.3982> 265.753.1-76.874.168.4-79.339.333.1-45.889.485.8-92.2
In-hospital mortality
CRS0.6751> 294.380.8-99.373.267.8-78.229.525.4-33.999.196.6-99.8
CTP0.5632> 794.380.8-99.362.056.2-67.622.820.0-25.898.996.0-99.7
AIMS650.6339> 180.063.1-91.683.478.6-87.536.429.6-43.797.294.8-98.6
MELD-Na0.5685> 1771.453.7-85.485.480.9-89.236.829.1-45.196.293.7-97.7
MELD0.5157> 1274.356.7-87.577.372.1-81.928.022.6-34.196.293.5-97.8
mGBS0.4349> 1068.650.7-83.174.969.6-79.824.519.4-30.495.392.5-97.1
CANUKA0.3521> 1245.728.8-63.489.585.4-92.734.024.0-45.893.391.1-95.0
GBS0.3700> 1274.356.7-87.562.756.9-68.219.115.6-23.295.492.1-97.3
In-hospital adverse outcomes
CTP0.4699> 778.868.6-86.968.261.9-73.946.241.0-51.590.385.9-93.4
CRS0.5095> 271.861.0-81.079.273.6-84.154.547.5-61.289.085.1-91.9
MELD-Na0.3758> 1369.458.5-79.068.261.9-73.943.137.5-48.886.582.2-89.9
MELD0.3241> 1345.935.0-57.086.581.6-90.554.244.4-63.682.279.0-84.9
AIMS650.3830> 151.840.7-62.786.581.6-90.557.147.7-66.183.880.5-86.6
In-hospital mortality

Thirty-five patients died during hospitalization. The in-hospital mortality rate was 10.6% in all patients, 25.4% in patients with in-hospital rebleeding and 6.8% in patients without in-hospital rebleeding (Table 2). Only one patient died of extrahepatic disease; the cause of death in the other patients was variceal bleeding or organ failure. The CRS, CTP, AIMS65, MELD-Na and MELD showed excellent discriminative ability; their AUROCs were greater than 0.8 and statistically significant (Figure 2B; Table 3). Furthermore, pairwise comparisons found no significant differences in these scoring systems. The AUROCs of mGBS, CANUKA and GBS were relatively small, but clinically acceptable (AUROCs > 0.7). The diagnostic value indices for predicting in-hospital mortality are presented in Table 4. Moreover, the calibration of all scoring systems was excellent (Table 3). The graphical analysis showed similar results for the CTP and CRS (Supplementary Figure 2).

In-hospital adverse outcomes

In-hospital adverse outcomes included rebleeding and death events. A total of 85 (25.8%) patients suffered from in-hospital adverse events (Table 2). The discriminative abilities of CTP, CRS, MELD-NA, MELD and AIMS65 were found to be clinically useful (AUROCs > 0.7) (Figure 2C; Table 3). Pairwise comparisons showed no significant differences among the CTP, CRS and MELD-Na scoring systems (P > 0.05). However, the CTP was superior to the MELD, AIMS65, CANUKA, GBS and mGBS in predicting in-hospital adverse outcomes (P < 0.05). The diagnostic value indices for predicting in-hospital adverse outcomes are presented in Table 4. The calibration of all scoring systems was good, except for the CRS (P < 0.05) (Table 3). The graphical analysis showed similar results for the CTP and CRS, but the CRS may underestimate the risk of adverse outcomes in high-risk strata (3-6) (Supplementary Figure 3).

Sensitivity analysis

In the sensitivity analysis of patients with esophageal variceal bleeding, the results were almost unchanged. The CTP and CRS remained the two best scoring systems for predicting in-hospital outcomes. For predicting in-hospital rebleeding, the AUROCs of the CTP and CRS were 0.75 (0.68-0.82) and 0.72 (0.64-0.80), respectively. For predicting in-hospital mortality and in-hospital adverse outcomes, the AUROCs of the CTP and CRS were 0.88 (0.83-0.94) and 0.89 (0.83-0.95), 0.81 (0.75-0.87) and 0.78 (0.71-0.85), respectively. In addition, the calibration of the CTP and CRS was good for predicting in-hospital rebleeding or mortality (P > 0.1). When focusing on patients who received endoscopic treatments, only the CTP was statistically significant for predicting in-hospital outcomes. The AUROC of the CTP was 0.70 (0.55-0.84) for predicting in-hospital rebleeding, 0.79 (0.63-0.94) for in-hospital mortality and 0.71 (0.57-0.84) for in-hospital adverse outcomes. The calibration of the CTP was good for predicting any in-hospital outcome (P > 0.1).

DISCUSSION

The results of the present study revealed that these scoring systems could effectively predict the occurrence of in-hospital adverse outcomes in cirrhotic patients with variceal bleeding. For in-hospital rebleeding, all scoring systems were able to distinguish whether in-hospital rebleeding occurred, and the calibration ability of these scores was good. However, only the CTP and CRS were clinically acceptable in terms of their discriminative ability. For in-hospital mortality, the CRS, CTP, AIMS65, MELD, and MELD-Na showed excellent discriminative and calibration abilities. The discriminative ability of the other prognostic scoring systems (GBS, mGBS and CANUKA) was slightly poor, but clinically acceptable.

Acute variceal bleeding is one of the most serious complications in patients with cirrhosis. This study found that the rate of in-hospital rebleeding and mortality could be as high as 20.3% and 10.6%, respectively, and the mortality rate was higher in patients with in-hospital rebleeding than in those without. These findings are similar to the results reported by previous studies[4,15]. Considering the harmfulness of variceal bleeding, appropriate risk stratification is critical for the optimal management of these patients. Close monitoring and aggressive treatment should be considered for high-risk patients. However, some ideal prognostic scores are controversial due to poor external validation. In fact, the CRS, AIMS65, CANUKA and GBS scoring systems were established independently with different study purposes and populations. The GBS was developed and used to predict UGIB patients’ risk of requiring blood transfusion or intervention, decreased hemoglobin, rebleeding and mortality. Similarly, the CANUKA was used to identify high-risk patients with 30-day rebleeding or death, radiologic or surgical intervention for bleeding control, and the need for therapeutic endoscopy or transfusion. Both the GBS and CANUKA are recommended for screening patients for hospital intervention or outpatient treatment[11,17]. The mGBS is similar to the GBS; it was developed by removing the subjective variables of the GBS[10]. In contrast, the CRS and AIMS65 were developed to determine the risk of in-hospital rebleeding or mortality in patients with UGIB[18,19]. All of the above prognostic scores included unselected UGIB as the research subject, and both patients with variceal bleeding and those with nonvariceal bleeding were enrolled for analysis. However, the proportion of patients with variceal bleeding was very low. Therefore, the predictive performance of these scores in predicting the risk of rebleeding or mortality for patients with variceal hemorrhage is unclear. Stanley et al[16] performed an international multicenter prospective study and found that the GBS was the best scoring system in predicting the need for intervention (transfusion, endoscopic treatment, interventional or surgical intervention) or death. According to their study, the latest guideline recommended that a GBS score of ≤ 1 could be used to identify low-risk patients for nonvariceal UGIB[20]. However, their study only included a few patients with variceal bleeding (7%) and did not perform a subgroup analysis for this group of patients. Gaduputi et al[21] reported that the AIMS65 score may be as useful as the Rockall score for predicting the risk of rebleeding and death in noncirrhotic patients. External validation studies confirmed that these scores had poorer predictive ability in patients with variceal bleeding than in those with nonvariceal bleeding[22,23].

For predicting in-hospital adverse outcomes, few studies have explored the usefulness of these scores in patients with variceal hemorrhage, and the conclusions are controversial. Motola-Kuba et al[9] found that the GBS was better at predicting in-hospital rebleeding than the Rockall score (RS), AIMS65, CTP and MELD. Sarwar et al[8] showed that the Rockall score had good discriminative value for predicting in-hospital rebleeding. However, Choe et al[7] and Jairath et al[4] asserted that the GBS, CRS and AIMS65 had limited ability for predicting the risk of in-hospital rebleeding, with AUROCs of approximately 0.6. The present study was performed in Chinese patients, included more comprehensive scoring systems, and found that these scores, apart from the CTP and CRS, had poor predictive ability. In addition, the AUROCs of the CTP and MELD were the same as those reported in a study from South Korea[24]. These differences among different studies are understandable as these studies were conducted in different countries or regions and enrolled patients with different characteristics. For predicting in-hospital mortality, the AUROCs of these scores also varied between different studies. Compared with predicting in-hospital rebleeding, previous studies have generally reported that these scores were better at predicting the risk of in-hospital death[8,9,24,25]. These findings are consistent with our results. In fact, the component variables of these scores indicate that they are more suitable for predicting short-term death rather than rebleeding. The variables included in these scores are age, systolic blood pressure, heart rate, hemoglobin, comorbidity, albumin, international normalized ratio and blood urea nitrogen (Supplementary Table 1). Previous studies have found that these variables were independently associated with short-term mortality in patients with cirrhosis[26,27]. In contrast, most of these variables have not been confirmed to be associated with short-term rebleeding. Some independent factors, such as ascites, portal vein thrombosis and portal hypertension, were identified to be associated with early variceal rebleeding, but they are not included in the validated scores[26,28,29]. In addition, many studies have shown that the CTP, MELD and MELD-Na have good predictive values in predicting in-hospital death, and our study confirms this finding[30].

This study has some strengths. First, a relatively adequate sample size helped to evaluate the predictive value of these scores. Second, most risk scoring systems lack external validation, especially in Chinese patients. Third, some of these prognostic scoring systems were validated for in-hospital adverse outcomes for the first time. Finally, our study provides some evidence in Chinese patients. However, some limitations should also be mentioned. First, the present study was a single-center retrospective study, and the applicability of the results may be limited. Second, not all patients were treated following the current guidelines. However, our data were obtained from clinical records and reflected real clinical practices. Third, some transferred patients were excluded because some of the data could change after external treatments, and the data from other hospitals were not available. Fourth, some endoscopy-based scores were not considered in this study because a risk assessment was delayed or even unachievable in some healthcare settings using an endoscopy score. Furthermore, most endoscopic data are subjective. In addition, some subgroup or sensitivity analyses could not be performed because some the relevant data were not available or the effective sample size was insufficient after patients were split into several groups. The predictive values of these prognostic scores require validation based on different degrees of portal hypertension, grading and types of varices, and types of special treatment. In conclusion, the risk of in-hospital rebleeding and mortality remains high in cirrhotic patients with variceal bleeding. The predictive value of the CTP and CRS are clinically acceptable for predicting in-hospital rebleeding. The performances of these scoring systems are better at predicting in-hospital mortality than in-hospital rebleeding, especially the CRS, CTP, AIMS65, MELD-Na and MELD. Further prospective and multicenter studies are warranted to confirm our findings.

ARTICLE HIGHLIGHTS
Research background

Several risk scoring systems have been developed and are regarded as useful tools for predicting clinical outcomes in patients with upper gastrointestinal bleeding (UGIB). As a common form of acute UGIB, patients with variceal bleeding often have an increased risk of in-hospital adverse outcomes. Data are limited regarding the predictive value of these risk scoring systems for patients with variceal bleeding.

Research motivation

Variceal bleeding is a serious complication of cirrhosis, and discovering valuable prognostic scores will be useful for early identification of high-risk patients. These patients will benefit if necessary measures are taken timely.

Research objectives

The present study aimed to validate the predictive value of eight scoring systems for in-hospital outcomes in cirrhotic patients with variceal bleeding.

Research methods

Consecutive patients with acute variceal bleeding, from March 2017 to June 2019, were included at the Second Affiliated Hospital of Xi’an Jiaotong University. By reviewing medical records, required data were collected and prognostic scores were calculated for the clinical Rockall score (CRS), AIMS65 score (AIMS65), Glasgow-Blatchford score (GBS), modified GBS (mGBS), Canada-United Kingdom-Australia score (CANUKA), Child-Turcotte-Pugh score (CTP), model for end-stage liver disease (MELD) and MELD-Na. The discriminative ability of these prognostic scores was assessed using the area under the receiver operating characteristic curve (AUROC), and the calibration was evaluated by the Hosmer-Lemeshow (H-L) test.

Research results

We retrospectively enrolled 330 cirrhotic patients with variceal bleeding. The rate of in-hospital rebleeding for these patients was 20.3%, and the rate of in-hospital mortality was 10.6%. For predicting in-hospital rebleeding, although all AUROCs of these prognostic scores were statistically significant, only the AUROCs of the CTP and CRS were clinically acceptable (AUROC > 0.7). The calibration of all prognostic scores for in-hospital rebleeding was good. For predicting in-hospital mortality, all AUROCs of these prognostic scores were good with statistical significance, especially the CRS, CTP, AIMS65, MELD-Na and MELD (AUROCs > 0.8). The calibration of all prognostic scores for in-hospital mortality was also good.

Research conclusions

The risk of in-hospital adverse outcomes remains high in cirrhotic patients with variceal bleeding. The CTP and CRS have acceptable abilities for predicting in-hospital rebleeding. All of these prognostic scores are useful for predicting in-hospital mortality, especially the CRS, CTP, AIMS65, MELD-Na and MELD. Clinicians from hospitals of different grades can select suitable models for early identification of high-risk patients.

Research perspectives

The predictive value of these prognostic scores still need to be confirmed in patients with special risk factors, such as gastric variceal bleeding, high portal pressure and those receiving special treatments. Predictive models with high accuracy need to be established for predicting in-hospital rebleeding taking into account the limitations of existing models.

Footnotes

Manuscript source: Unsolicited manuscript

Specialty type: Gastroenterology and hepatology

Country of origin: China

Peer-review report classification

Grade A (Excellent): A

Grade B (Very good): B, B

Grade C (Good): C, C

Grade D (Fair): 0

Grade E (Poor): 0

P-Reviewer: Ali FEM, Dourakis SP, El-Bendary M, Garbuzenko D, Gencdal G S-Editor: Gong ZM L-Editor: Webster JR E-Editor: Ma YJ

References
1.  European Association for the Study of the Liver. EASL Clinical Practice Guidelines for the management of patients with decompensated cirrhosis. J Hepatol. 2018;69:406-460.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 660]  [Cited by in F6Publishing: 452]  [Article Influence: 220.0]  [Reference Citation Analysis (0)]
2.  Tsochatzis EA, Bosch J, Burroughs AK. Liver cirrhosis. Lancet. 2014;383:1749-1761.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 801]  [Cited by in F6Publishing: 358]  [Article Influence: 114.4]  [Reference Citation Analysis (0)]
3.  Tripathi D, Stanley AJ, Hayes PC, Patch D, Millson C, Mehrzad H, Austin A, Ferguson JW, Olliff SP, Hudson M, Christie JM; Clinical Services and Standards Committee of the British Society of Gastroenterology. U.K. guidelines on the management of variceal haemorrhage in cirrhotic patients. Gut. 2015;64:1680-1704.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 260]  [Cited by in F6Publishing: 172]  [Article Influence: 43.3]  [Reference Citation Analysis (1)]
4.  Jairath V, Rehal S, Logan R, Kahan B, Hearnshaw S, Stanworth S, Travis S, Murphy M, Palmer K, Burroughs A. Acute variceal haemorrhage in the United Kingdom: patient characteristics, management and outcomes in a nationwide audit. Dig Liver Dis. 2014;46:419-426.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 54]  [Cited by in F6Publishing: 41]  [Article Influence: 7.7]  [Reference Citation Analysis (0)]
5.  Ebrahimi Bakhtavar H, Morteza Bagi HR, Rahmani F, Shahsavari Nia K, Ettehadi A. Clinical Scoring Systems in Predicting the Outcome of Acute Upper Gastrointestinal Bleeding; a Narrative Review. Emerg (Tehran). 2017;5:e36.  [PubMed]  [DOI]  [Cited in This Article: ]
6.  Monteiro S, Gonçalves TC, Magalhães J, Cotter J. Upper gastrointestinal bleeding risk scores: Who, when and why? World J Gastrointest Pathophysiol. 2016;7:86-96.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in CrossRef: 23]  [Cited by in F6Publishing: 11]  [Article Influence: 4.6]  [Reference Citation Analysis (0)]
7.  Choe JW, Kim SY, Hyun JJ, Jung SW, Jung YK, Koo JS, Yim HJ, Lee SW. Is the AIMS 65 Score Useful in Prepdicting Clinical Outcomes in Korean Patients with Variceal and Nonvariceal Upper Gastrointestinal Bleeding? Gut Liver. 2017;11:813-820.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 6]  [Article Influence: 2.7]  [Reference Citation Analysis (0)]
8.  Sarwar S, Dilshad A, Khan AA, Alam A, Butt AK, Tariq S, Ahmad I. Predictive value of Rockall score for rebleeding and mortality in patients with variceal bleeding. J Coll Physicians Surg Pak. 2007;17:253-256.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in F6Publishing: 6]  [Reference Citation Analysis (0)]
9.  Motola-Kuba M, Escobedo-Arzate A, Tellez-Avila F, Altamirano J, Aguilar-Olivos N, González-Angulo A, Zamarripa-Dorsey F, Uribe M, Chávez-Tapia NC. Validation of prognostic scores for clinical outcomes in cirrhotic patients with acute variceal bleeding. Ann Hepatol. 2016;15:895-901.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in F6Publishing: 2]  [Reference Citation Analysis (0)]
10.  Cheng DW, Lu YW, Teller T, Sekhon HK, Wu BU. A modified Glasgow Blatchford Score improves risk stratification in upper gastrointestinal bleed: a prospective comparison of scoring systems. Aliment Pharmacol Ther. 2012;36:782-789.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 50]  [Cited by in F6Publishing: 45]  [Article Influence: 5.6]  [Reference Citation Analysis (0)]
11.  Oakland K, Kahan BC, Guizzetti L, Martel M, Bryant RV, Brahmania M, Singh S, Nguyen NQ, Sey MSL, Barkun A, Jairath V. Development, Validation, and Comparative Assessment of an International Scoring System to Determine Risk of Upper Gastrointestinal Bleeding. Clin Gastroenterol Hepatol. 2019;17:1121-1129.e2.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 12]  [Cited by in F6Publishing: 6]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
12.  Biselli M, Gramenzi A, Lenzi B, Dall'Agata M, Pierro ML, Perricone G, Tonon M, Bellettato L, D'Amico G, Angeli P, Boffelli S, Bonavita ME, Domenicali M, Caraceni P, Bernardi M, Trevisani F. Development and Validation of a Scoring System That Includes Corrected QT Interval for Risk Analysis of Patients With Cirrhosis and Gastrointestinal Bleeding. Clin Gastroenterol Hepatol. 2019;17:1388-1397.e1.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 6]  [Cited by in F6Publishing: 3]  [Article Influence: 2.0]  [Reference Citation Analysis (0)]
13.  Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 605]  [Cited by in F6Publishing: 504]  [Article Influence: 100.8]  [Reference Citation Analysis (0)]
14.  Hepatology Branch of Chinese Medical Association. Gastroenterology Branch of Chinese Medical Association; Endoscopy Branch of Chinese Medical Association. Guideline for the diagnosis and treatment of esophageal and gastric variceal bleeding in cirrhotic portal hypertension. Zhongguo Ganzangbing Zazhi. 2016;8:1-18.  [PubMed]  [DOI]  [Cited in This Article: ]
15.  Sanders DS, Carter MJ, Goodchap RJ, Cross SS, Gleeson DC, Lobo AJ. Prospective validation of the Rockall risk scoring system for upper GI hemorrhage in subgroups of patients with varices and peptic ulcers. Am J Gastroenterol. 2002;97:630-635.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 53]  [Cited by in F6Publishing: 52]  [Article Influence: 2.8]  [Reference Citation Analysis (0)]
16.  Stanley AJ, Laine L, Dalton HR, Ngu JH, Schultz M, Abazi R, Zakko L, Thornton S, Wilkinson K, Khor CJ, Murray IA, Laursen SB; International Gastrointestinal Bleeding Consortium. Comparison of risk scoring systems for patients presenting with upper gastrointestinal bleeding: international multicentre prospective study. BMJ. 2017;356:i6432.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 112]  [Cited by in F6Publishing: 77]  [Article Influence: 28.0]  [Reference Citation Analysis (0)]
17.  Blatchford O, Murray WR, Blatchford M. A risk score to predict need for treatment for upper-gastrointestinal haemorrhage. Lancet. 2000;356:1318-1321.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 548]  [Cited by in F6Publishing: 148]  [Article Influence: 26.1]  [Reference Citation Analysis (0)]
18.  Rockall TA, Logan RF, Devlin HB, Northfield TC. Risk assessment after acute upper gastrointestinal haemorrhage. Gut. 1996;38:316-321.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 809]  [Cited by in F6Publishing: 586]  [Article Influence: 32.4]  [Reference Citation Analysis (0)]
19.  Saltzman JR, Tabak YP, Hyett BH, Sun X, Travis AC, Johannes RS. A simple risk score accurately predicts in-hospital mortality, length of stay, and cost in acute upper GI bleeding. Gastrointest Endosc. 2011;74:1215-1224.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 223]  [Cited by in F6Publishing: 148]  [Article Influence: 22.3]  [Reference Citation Analysis (0)]
20.  Barkun AN, Almadi M, Kuipers EJ, Laine L, Sung J, Tse F, Leontiadis GI, Abraham NS, Calvet X, Chan FKL, Douketis J, Enns R, Gralnek IM, Jairath V, Jensen D, Lau J, Lip GYH, Loffroy R, Maluf-Filho F, Meltzer AC, Reddy N, Saltzman JR, Marshall JK, Bardou M. Management of Nonvariceal Upper Gastrointestinal Bleeding: Guideline Recommendations From the International Consensus Group. Ann Intern Med. 2019;.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 90]  [Cited by in F6Publishing: 24]  [Article Influence: 45.0]  [Reference Citation Analysis (0)]
21.  Gaduputi V, Abdulsamad M, Tariq H, Rafeeq A, Abbas N, Kumbum K, Chilimuri S. Prognostic Value of AIMS65 Score in Cirrhotic Patients with Upper Gastrointestinal Bleeding. Gastroenterol Res Pract. 2014;2014:787256.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 4]  [Cited by in F6Publishing: 4]  [Article Influence: 0.6]  [Reference Citation Analysis (0)]
22.  Thanapirom K, Ridtitid W, Rerknimitr R, Thungsuk R, Noophun P, Wongjitrat C, Luangjaru S, Vedkijkul P, Lertkupinit C, Poonsab S, Ratanachu-ek T, Hansomburana P, Pornthisarn B, Thongbai T, Mahachai V, Treeprasertsuk S. Prospective comparison of three risk scoring systems in non-variceal and variceal upper gastrointestinal bleeding. J Gastroenterol Hepatol. 2016;31:761-767.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 21]  [Cited by in F6Publishing: 12]  [Article Influence: 4.2]  [Reference Citation Analysis (0)]
23.  Rout G, Sharma S, Gunjan D, Kedia S, Nayak B, Shalimar. Comparison of various prognostic scores in variceal and non-variceal upper gastrointestinal bleeding: A prospective cohort study. Indian J Gastroenterol. 2019;38:158-166.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 3]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
24.  Kim GH, Kim JH, Kim YJ, Ko SY, Choe WH, Kwon SY, Lee CH. Value of the APASL severity score in patients with acute variceal bleeding: a single center experience. Hepatol Int. 2013;7:1058-1064.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1]  [Cited by in F6Publishing: 1]  [Article Influence: 0.1]  [Reference Citation Analysis (0)]
25.  Wong MW, Chen MJ, Chen HL, Kuo YC, Lin IT, Wu CH, Lee YK, Cheng CH, Bair MJ. Application of chronic liver failure-sequential organ failure assessment score for the predication of mortality after esophageal variceal hemorrhage post endoscopic ligation. PLoS One. 2017;12:e0182529.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 3]  [Cited by in F6Publishing: 1]  [Article Influence: 0.8]  [Reference Citation Analysis (0)]
26.  Krige JE, Kotze UK, Distiller G, Shaw JM, Bornman PC. Predictive factors for rebleeding and death in alcoholic cirrhotic patients with acute variceal bleeding: a multivariate analysis. World J Surg. 2009;33:2127-2135.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 29]  [Cited by in F6Publishing: 19]  [Article Influence: 2.4]  [Reference Citation Analysis (0)]
27.  Jepsen P. Comorbidity in cirrhosis. World J Gastroenterol. 2014;20:7223-7230.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in CrossRef: 33]  [Cited by in F6Publishing: 23]  [Article Influence: 5.5]  [Reference Citation Analysis (0)]
28.  Xu L, Ji F, Xu QW, Zhang MQ. Risk factors for predicting early variceal rebleeding after endoscopic variceal ligation. World J Gastroenterol. 2011;17:3347-3352.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in CrossRef: 18]  [Cited by in F6Publishing: 13]  [Article Influence: 1.8]  [Reference Citation Analysis (0)]
29.  Qi X, Su C, Ren W, Yang M, Jia J, Dai J, Xu W, Guo X. Association between portal vein thrombosis and risk of bleeding in liver cirrhosis: A systematic review of the literature. Clin Res Hepatol Gastroenterol. 2015;39:683-691.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 20]  [Cited by in F6Publishing: 13]  [Article Influence: 3.3]  [Reference Citation Analysis (0)]
30.  Levesque E, Hoti E, Azoulay D, Ichaï P, Habouchi H, Castaing D, Samuel D, Saliba F. Prospective evaluation of the prognostic scores for cirrhotic patients admitted to an intensive care unit. J Hepatol. 2012;56:95-102.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 116]  [Cited by in F6Publishing: 95]  [Article Influence: 11.6]  [Reference Citation Analysis (0)]