Multivariable prognostic models for post-hepatectomy liver failure: An updated systematic review

Table 2 Description of model development, performance, and evaluation within included studies

Ref.	Model development						Model performance			Model evaluation				Clinical effectiveness
	Purpose	Blind edpredictor assessment	Predictor selection for modelling	Predictor selection during modelling	Missing data	Statistical modelling method	Model calibration performed	Model discrimination assessment	AUC (95%CI) (training model)	Internalvalidity assessment	AUC (95%CI)	Externalvalidity assessment	AUC (95%CI)
Model development only
Fang et al[21]	Binary	NR	Univariate association	NR	NR	MLR	Yes, calibration plots	Yes, AUC and CI	0.845 (0.806-0.884)	Yes, split sample	0.854 (0.782-0.926)	NR	NR	No study
Meng et al[22]	Binary	NR	Univariate association	Stepwise	Yes	MLR	Yes, calibration plots	Yes, AUC only	0.697	Yes, bootstraps	0.668	NR	NR	No study
Peng et al[23]	Binary	NR	Univariate association	NR	NR	MLR	NR	Yes, AUC and CI	0.867 (0.790-0.943)	NR	NR	NR	NR	No study
Shi et al[24]	Binary	NR	Univariate association	Forward	Yes	MLR	NR	Yes, AUC and CI	0.844 (0.801-0.887)	NR	NR	NR	NR	Developed online calculators
Choi et al[25]	Binary	NR	A bootstrap resampling approach	Backward	Yes	MLR	Yes, calibration curve and HL test	Yes, AUC and CI	0.737 (0.687-0.787)	Yes, split sample	0.672 (0.577-0.767)	NR	NR	No study
Xu et al[26]	Binary	NR	NR	NR	Yes	Improved based on original model	NR	Yes, AUC and CI	0.772 (0.716-0.822)	NR	NR	NR	NR	No study
Mai et al[27]	Binary	NR	Univariate association	NR	Yes	ANN	Yes, calibration plots and HL test	Yes, AUC and CI	0.88 (0.836-0.925)	Yes, split sample	0.876 (0.801-0.950)	NR	NR	No study
Yugawa et al[28]	Binary	NR	Univariate association	NR	Yes	MLR	NR	Yes, AUC only	0.88	NR	NR	NR	NR	No study
Zhu et al[29]	Binary	NR	Univariate association and LASSO regression	Forward	NR	MLR	Yes, calibration curve and HL test	Yes, AUC and CI	0.894 (0.823-0.964)	NR	NR	NR	NR	No study
Lee et al[30]	Binary	NR	Univariate association	Backward	Yes	MLR	Yes, HL test	Yes, AUC and CIs	0.853 (0.802-0.904)	Yes, CV and bootstraps	0.852 (0.795-0.910)	NR	NR	No study
Cho et al[31]	Binary	NR	Univariate association with the Kaplan-Meier plots	Backward	Yes	Cox regression	NR	Yes, AUC with DeLong test	0.877 (0.805-0.948)	Yes, CV	0.8	NR	NR	No study
Li et al[32]	Binary	NR	Univariate association	Backward	Yes	MLR	Yes, calibration plots and HL test	Yes, AUC and CI	0.726 (0.696-0.760)	Yes, bootstraps	0.717 (0.663-0.770)	NR	NR	No study
Zhang et al[33]	Binary	NR	Univariate association	Forest algorithm	Yes	MLR	Yes, calibration plots	Yes, AUC and CI	0.773 (0.729-0.818)	NR	NR	NR	NR	No study
Wang et al[34]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration curve	Yes, AUC and CI	0.857 (0.789-0.925)	Yes, split sample	0.753 (0.696-0.809)	NR	NR	No study
Prodeau et al[35]	Binary	NR	Bivariate ordinal logistic regression model	Backward	Yes	MLR	Yes, Lipsitz and Pulkstenis-Robinson tests	Yes, AUC only	0.77	Yes, bootstraps	0.85	NR	NR	No study
Xiang et al[36]	Binary	NR	Univariate association and LASSO regression	NR	Yes	MLR	Yes, calibration plots and HL test	Yes, AUC and CI	0.842 (0.761-0.922)	Yes, split sample	0.863 (0.750-0.975)	NR	NR	No study
Zhong et al[37]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration plots	Yes, AUC and CI	0.832 (0.777-0.886)	Yes, split sample	0.803 (0.723-0.883)	NR	NR	No study
Chin et al[38]	Binary	NR	Univariate association	LASSO method	Yes	Penalized logistic regression	Yes, HL test	Yes, AUC only	0.823	NR	NR	NR	NR	No study
Morino et al[39]	Binary	NR	Univariate association	Stepwise	NR	MLR	NR	NR	0.877	NR	NR	NR	NR	No study
Wang et al[40]	Binary	NR	SHAP analysis	NR	Yes	ML	NR	Yes, AUC only	0.944 (0.924-0.964)	Yes, split sample	0.870 (0.791-0.950)	No external validity, just a split sample like Internal validity	NR	No study
Dhir et al[41]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration plots and HL test	Yes, AUC only	0.78	Yes, split sample	0.78	NR	NR	No study
Model development and external validation
Lei et al[42]	Binary	NR	The squares of the Spearman correlation coefficients	LASSO method	Yes	MLR	Yes, calibration curve	Yes, AUC with DeLong test	0.73 (0.69-0.76)	Yes, CV (10-fold)	0.73 (0.69-0.76)	Different patient population (other two hospitals)	0.72 (0.65-0.78)	No study
Xu et al[43]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration curve	Yes, AUC only	0.863 (0.812-0.914)	Yes, split sample	0.823 (0.737-0.909)	Different patient population	0.74 (0.624-0.856)	No study
Peng et al[44]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration curve and HL test	Yes, AUC with Delong test	0.828 (0.756-0.901)	NR	NR	Different patient population	0.821 (0.727-0.914)	No study
Ye et al[45]	Binary	NR	Univariate association	Forest algorithm	Yes	MLR	Yes, calibration plots	Yes, AUC and CI	0.868 (0.811-0.926)	Yes, split sample	0.868 (0.811-0.926)	Different patient population	0.82 (0.756-0.861)	Yes
Hobeika et al[46]	Binary	NR	Binomial logistic regressions	Forward/backward	Yes	MLR	Yes, calibration plots and HL test	Yes, AUC with DeLong test	0.77 (0.667, 0.872)	NR	NR	Different patient population	0.888 (0.809-0.968)	No study
Li et al[47]	Binary	NR	Univariate association	Forward	Yes	MLR	Yes, calibration curve	Yes, AUC only	0.911 (0.865-0.958)	NR	NR	Different time period	0.714 (0.697-0.902)	No study
Chen et al[50]	Binary	NR	Univariate association; Pearson’s correlation coefficients	NR	Yes	MLR	NR	Yes, AUC only	0.956 (0.955-0.962)	NR	NR	Different patient population	0.844 (0.833-0.886)	No study
Shen et al[51]	Binary	NR	Univariate association	NR	NR	MLR	Yes, calibration curve	Yes, AUC and CI	0.818 (0.735-0.901)	NR	NR	Different patient population	0.906 (0.833-0.979)	No study
Ding et al[52]	Binary	NR	Univariate association	NR	Yes	MLR	Yes, calibration curve and HL test	Yes, AUC only	0.91	Yes, split sample	0.82	Different patient population	0.89	No study
Xu et al[53]	Binary	NR	LASSO regression with 10-fold cross-validation	LASSO method	NR	MLR	Yes, calibration curve	Yes, AUC with Delong test	0.838 (0.790-0.885)	Yes, split sample	0.788 (0.693-0.884)	Different patient population	0.750 (0.632-0.868)	No study
Wang et al[54]	Binary Mortality + OS	NR	Univariate association	Backward	NR	MLR	Yes, calibration plots	Yes, AUC only	0.883 (0.852-0.915)	Yes, split sample	0.851	Different patient population	0.856	No study
External validation of pre-existing model
Guo et al[55]	Binary	NR	NR	NR	Yes	NR	Yes, loess-smoothed plots	Yes, AUC with DeLong test	NR	NR	NR	NR	0.64 (0.58-0.69); 0.58 (0.52-0.64); 0.59 (0.53-0.64); 0.57 (0.51-0.63); 0.57 (0.51-0.63); 0.61 (0.55-0.67)	No study
Noji et al[56]	Binary	NR	NR	NR	Yes	NR	NR	Yes, AUC only	NR	NR	NR	NR	0.62	No study

NR: Not reported; MLR: Multivariable logistic regression; ANN: Artificial neural network; ML: Machine learning; LASSO method: Least absolute contraction selection operator logistic regression; HL: Hosmer-Leme show test of goodness-of-fit; AUC: Area under the receiver operating characteristic curve; CV: Cross validation.