Development and validation of a postoperative prediction model for residual stones after hepatectomy in patients with hepatolithiasis

Table 1 Cohort partition and analytical role

Cohort	Patients (n)	Events (n)	Non-events (n)	Event rate (%)	Analytical role
Overall analytic cohort	300	82	218	27.3	Source population for development and internal validation
Development cohort	211	58	153	27.5	Repeated nested-CV pipeline comparison and selection; full-development fitting of the selected pipeline; OOF threshold derivation
Validation cohort	89	24	65	27.0	Untouched until one-time evaluation of the locked model and threshold

OOF: Out-of-fold; CV: Cross-validation.

Table 2 Clinically relevant characteristics according to residual-stone status, n (%)/median (interquartile range)

Item	Overall (n = 300)	Non-residual stone group (n = 218)	Residual stone group (n = 82)	P value
Baseline characteristics
Age (years)	58.00 (51.00-66.00)	58.00 (52.00-66.00)	56.50 (50.00-65.00)	0.401
Sex				0.428
Male	101 (33.67)	70 (32.11)	31 (37.80)
Female	199 (66.33)	148 (67.89)	51 (62.20)
BMI	22.04 (19.94-24.22)	22.22 (19.98-24.41)	21.34 (19.55-23.56)	0.195
Symptoms
Abdominal pain	229 (76.33)	161 (73.85)	68 (82.93)	0.135
Nausea and vomiting	71 (23.67)	46 (21.10)	25 (30.49)	0.121
Fever	72 (24.00)	46 (21.10)	26 (31.71)	0.078
Jaundice	27 (9.00)	19 (8.72)	8 (9.76)	0.957
HBV infection	34 (11.33)	23 (10.55)	11 (13.41)	0.622
Diabetes	43 (14.33)	24 (11.01)	19 (23.17)	0.013
Hypertension	49 (16.33)	39 (17.89)	10 (12.20)	0.311
Previous surgical history
Cholecystectomy	114 (38.00)	77 (35.32)	37 (45.12)	0.154
Choledochoscopy	64 (21.33)	45 (20.64)	19 (23.17)	0.750
Cholangioenterostomy	19 (6.33)	14 (6.42)	5 (6.10)	1.000
Partial hepatectomy	14 (4.67)	8 (3.67)	6 (7.32)	0.304
Stone classification1				0.289
Type I	34 (11.33)	28 (12.84)	6 (7.32)
Type IIa	54 (18.00)	39 (17.89)	15 (18.29)
Type IIb	48 (16.00)	39 (17.89)	9 (10.98)
Type IIc	10 (3.33)	7 (3.21)	3 (3.66)
Type E	154 (51.33)	105 (48.17)	49 (59.76)
Disease duration (months)	1.50 (0.50-12.00)	1.50 (0.50-12.00)	1.50 (0.50-21.00)	0.625
Imaging
Gallstones	77 (25.67)	58 (26.61)	19 (23.17)	0.646
Cholangitis	117 (39.00)	79 (36.24)	38 (46.34)	0.143
Liver abscess	44 (14.67)	27 (12.39)	17 (20.73)	0.101
Liver atrophy	104 (34.67)	74 (33.94)	30 (36.59)	0.770
Liver cirrhosis	23 (7.67)	12 (5.50)	11 (13.41)	0.040
Biliary stricture	79 (26.33)	33 (23.24)	46 (29.11)	0.307
Biliary dilation	282 (94.00)	205 (94.04)	77 (93.90)	1.000
Stone location				< 0.001
Unilateral	225 (75.00)	183 (83.94)	42 (51.22)
Bilateral	75 (25.00)	35 (16.06)	40 (48.78)
Intraoperative characteristics
Surgical method				0.486
OH	142 (47.33)	100 (45.87)	42 (51.22)
LH	158 (52.67)	118 (54.13)	40 (48.78)
Operative time (minutes)	295.00 (230.75-335.00)	280.00 (219.25-322.75)	335.00 (305.50-404.00)	< 0.001
Intraoperative blood loss (mL)	300.00 (200.00-500.00)	300.00 (200.00-500.00)	400.00 (200.00-600.00)	0.009
Intraoperative blood transfusion	26 (8.67)	17 (7.80)	9 (10.98)	0.521
ASA classification				0.505
I	205 (68.33)	146 (66.97)	59 (71.95)
II	93 (31.00)	71 (32.57)	22 (26.83)
III	2 (0.67)	1 (0.46)	1 (1.22)
Number of pringle maneuvers				0.220
≤ 3	215 (71.67)	161 (73.85)	54 (65.85)
> 3	85 (28.33)	57 (26.15)	28 (34.15)
Liver resection extent				0.202
Left	158 (52.67)	118 (54.13)	40 (48.78)
Left lateral	71 (23.67)	55 (25.23)	16 (19.51)
Right	34 (11.33)	20 (9.17)	14 (17.07)
Right posterior	15 (5.00)	8 (3.67)	7 (8.54)
Left medial lobe + right anterior	5 (1.67)	4 (1.83)	1 (1.22)
Right anterior	3 (1.00)	3 (1.38)	0 (0.00)
Others2	14 (4.67)	10 (4.59)	4 (4.88)
Other related procedures
Cholecystectomy	136 (45.33)	97 (44.50)	39 (47.56)	0.730
Choledochoenterostomy	90 (30.00)	61 (27.98)	29 (35.37)	0.270
T-tube drainage	210 (70.00)	168 (77.06)	42 (51.22)	< 0.001
Postoperative complications
Anastomotic stenosis	30 (10.00)	16 (7.34)	14 (17.07)	0.022
Postoperative biliary stricture	56 (18.67)	20 (9.17)	36 (43.90)	< 0.001
Abdominal infection	23 (7.67)	17 (7.80)	6 (7.32)	1.000
Pulmonary infection	93 (31.00)	71 (32.57)	22 (26.83)	0.413
Incision infection	12 (4.00)	7 (3.21)	5 (6.10)	0.420
Abdominal hemorrhage	8 (2.67)	5 (2.29)	3 (3.66)	0.801
Intestinal obstruction	6 (2.00)	3 (1.38)	3 (3.66)	0.426
Positive pathogen culture
Bile	57 (19.00)	32 (14.68)	25 (30.49)	0.003
Peritoneal drainage fluid	27 (9.00)	14 (6.42)	13 (15.85)	0.020
Blood	32 (10.67)	21 (9.63)	11 (13.41)	0.462
Bile leakage	38 (12.67)	16 (7.34)	22 (26.83)	< 0.001
Intestinal leakage	11 (3.67)	4 (1.83)	7 (8.54)	0.016

¹Stone classification was based on Dong’s classification.

²Others indicate single-segment resections grouped because of small sample sizes.

BMI: Body mass index; HBV: Hepatitis B virus; OH: Open hepatectomy; LH: Laparoscopic hepatectomy; ASA: American Society of Anesthesiologists.

Table 3 Comparative performance of the 12 candidate models based on development-cohort out-of-fold predictions

Candidate algorithm	AUC	Brier score	Log loss	Calibration intercept	Calibration slope
LASSO-LR	0.849	0.138	0.432	0.022	1.748
XGBoost	0.817	0.153	0.469	0.227	0.705
GBM	0.813	0.150	0.457	0.076	0.938
RF	0.803	0.154	0.472	0.016	1.085
CatBoost	0.801	0.167	0.518	0.510	0.589
LDA	0.796	0.171	0.565	0.122	0.434
NB	0.788	0.187	0.689	0.937	0.334
RBF-SVM	0.785	0.158	0.481	-0.055	0.939
PLSR	0.784	0.175	0.535	-0.581	2.237
LightGBM	0.780	0.183	0.679	0.855	0.338
ANN	0.757	0.172	0.568	0.047	0.475
k-NN	0.743	0.180	0.596	0.723	0.709

Values were derived from patient-level out-of-fold predictions generated via 15 outer resampled iterations within the development cohort (n = 211). Bold formatting highlights the pre-specified primary model. LASSO-LR: Least absolute shrinkage and selection operator-logistic regression; XGBoost: EXtreme Gradient Boosting; GBM: Gradient boosting machine; RF: Random forest; CatBoost: Categorical boosting; LDA: Linear discriminant analysis; NB: Naive Bayes; RBF-SVM: Support vector machine with radial basis function kernel; PLSR: Partial least squares regression; LightGBM: Light gradient boosting machine; ANN: Artificial neural network; k-NN: K-nearest neighbors; AUC: Area under the receiver operating characteristic curve.

Table 4 Performance of the least absolute shrinkage and selection operator-logistic regression model in development and held-out validation cohorts

Metric	Development OOF estimate (95%CI)	Held-out validation estimate (95%CI)
AUC	0.849 (0.794-0.904)	0.877 (0.807-0.947)
Brier score	0.138 (0.113-0.163)	0.134 (0.098-0.171)
Log loss	0.432 (0.375-0.492)	0.417 (0.335-0.499)
Calibration intercept	0.022 (-0.288 to 0.318)	0.006 (-0.543 to 0.467)
Calibration slope	1.748 (1.365-2.315)	1.939 (1.353-3.107)

Development estimates were calculated from patient-level out-of-fold predictions across 15 outer resampled iterations in the development cohort (n = 211). Held-out validation estimates were evaluated in a single pass on the frozen model using the independent validation cohort (n = 89). 95% confidence interval was derived via 1000 patient-level non-parametric bootstrap iterations. AUC: Area under the receiver operating characteristic curve; CI: Confidence interval; OOF: Out-of-fold.

Table 5 Classification performance at the locked threshold of 0.270

Metric	Development OOF (95%CI)	Held-out validation (95%CI)
Sensitivity	0.776 (0.661-0.882)	0.875 (0.727-1.000)
Specificity	0.784 (0.715-0.845)	0.785 (0.677-0.879)
PPV	0.577 (0.468-0.688)	0.600 (0.440-0.759)
NPV	0.902 (0.846-0.951)	0.944 (0.873-1.000)
Accuracy	0.782 (0.725-0.839)	0.809 (0.719-0.888)
F1 score	0.662 (0.563-0.750)	0.712 (0.571-0.833)
Youden index	0.560 (0.433-0.683)	0.660 (0.470-0.820)

Data are presented as point estimates with 95% confidence intervals in parentheses. The locked threshold of 0.270 was derived from the maximum Youden index in the development cohort via 10-fold cross-validation. CI: Confidence interval; OOF: Out-of-fold; NPV: Negative predictive value; PPV: Positive predictive value.