Endoscopic ultrasound-based deep learning for predicting chemotherapy response in unresectable pancreatic ductal adenocarcinoma

Table 1 Baseline clinical characteristics of patients stratified by RECIST version 1.1-based chemotherapy response groups

Characteristics	Non-PD	PD	P value
Age, years	59.65 ± 8.26	58.90 ± 10.09	0.626
Female	62 (41.6)	11 (26.8)	0.123
T category			0.500
T1	4 (2.7)	1 (2.4)
T2	8 (5.4)	1 (2.4)
T3	41 (27.5)	16 (39.0)
T4	96 (64.4)	23 (56.1)
N category			0.166
N0	21 (14.1)	3 (7.3)
N1	114 (76.5)	30 (73.2)
N2	10 (6.7)	7 (17.1)
N3	4 (2.7)	1 (2.4)
M category			0.588
M0	60 (40.3)	14 (34.1)
M1	89 (59.7)	27 (65.9)
Clinical stage			0.906
I	3 (2.0)	1 (2.4)
II	11 (7.4)	3 (7.3)
III	45 (30.2)	10 (24.4)
IV	90 (60.4)	27 (65.9)
CA19-9, U/mL	1908.72 ± 4357.01	1884.20 ± 4388.78	0.975
Tumor size, mm	47.59 ± 18.93	42.05 ± 17.40	0.081
Chemotherapy			0.259
mFOLFIRINOX	39 (26.2)	16 (39.0)
Gemcitabine + nab-PTX	66 (44.3)	14 (34.1)
Other regimens	44 (29.5)	11 (26.8)

Data are presented as n (%) or mean ± SD. Patients were classified into non-progressive disease (PD) and PD groups according to the RECIST version 1.1 response classification, which is distinct from the model-derived high-risk and low-risk groups used in the survival analysis (defined by the median patient-level risk score, with 95 patients in each group). The overall cohort median age (62 years, interquartile range 55–69 years) is reported before response-group stratification, whereas group-specific age is reported as mean ± SD. The P value for chemotherapy regimen distribution reflects the comparison between the non-PD and PD groups using the χ² test or Fisher’s exact test, as appropriate. CA19-9: Carbohydrate antigen 19-9; mFOLFIRINOX: Modified FOLFIRINOX; Non-PD: Disease control (complete response, partial response, or stable disease as per RECIST version 1.1); PD: Progressive disease (increase in target lesion size ≥ 20% or appearance of new lesions as per RECIST version 1.1); PTX: Paclitaxel.

Table 2 Performance metrics of the VGG19 model for chemotherapy response prediction in the training, internal validation, and independent test cohorts

Group	AUC (95%CI)	ACC (%) (95%CI)	SEN (%) (95%CI)	SPE (%) (95%CI)	PPV (%)	NPV (%)	FNR (%)	F1
Train	0.963	91.11	82.81	92.73	69.01	96.50	17.19	0.753
Val	0.758	79.00	58.25	83.06	40.19	91.05	41.75	0.476
Test	0.614 (0.532–0.692)	71.58 (67.84–75.31)	36.54 (22.91–50.00)	75.81 (71.82–79.49)	15.45	90.81	63.46	0.217

95% confidence intervals (CIs) for area under the receiver operating characteristic curve were computed using the DeLong method, whereas 95%CIs for accuracy, sensitivity, and specificity were computed using the Wilson score interval method. CIs are reported for the independent test cohort only. AUC: Area under the receiver operating characteristic curve; ACC: Accuracy; CI: Confidence interval; F1: F1-score; FNR: False-negative rate; NPV: Negative predictive value; PPV: Positive predictive value; SEN: Sensitivity; SPE: Specificity; Test: Independent test cohort; Train: Training cohort; Val: Internal validation cohort.

Table 3 Performance metrics of the VGG19-BN model for chemotherapy response prediction in the training, validation, and independent test cohorts

Group	AUC (95%CI)	ACC (%) (95%CI)	SEN (%) (95%CI)	SPE (%) (95%CI)	PPV (%)	NPV (%)	FNR (%)	F1
Train	1.000	99.77	100.00	99.73	98.622	100.00	0	0.993
Val	0.864	90.36	66.67	94.99	72.24	93.58	33.33	0.693
Test	0.775 (0.705–0.841)	82.78 (79.05–85.89)	46.15 (32.43–59.02)	87.21 (83.72–90.19)	30.38	93.05	53.85	0.366

95% confidence intervals (CIs) for area under the receiver operating characteristic curve were computed using the DeLong method, whereas 95%CIs for accuracy, sensitivity, and specificity were computed using the Wilson score interval method. CIs are reported for the independent test cohort only. AUC: Area under the receiver operating characteristic curve; ACC: Accuracy; CI: Confidence interval; F1: F1-score; FNR: False-negative rate; NPV: Negative predictive value; PPV: Positive predictive value; SEN: Sensitivity; SPE: Specificity; Train: Training cohort; Test: Independent test cohort; Val: Internal validation cohort.

Table 4 Performance metrics of the ResNet50 model for chemotherapy response prediction in the training, internal validation, and independent test cohorts

Group	AUC (95%CI)	ACC (%) (95%CI)	SEN (%) (95%CI)	SPE (%) (95%CI)	PPV (%)	NPV (%)	FNR (%)	F1
Train	0.962	92.71	79.30	95.34	76.87	95.93	20.70	0.781
Val	0.871	83.65	76.49	85.05	50.00	94.87	23.51	0.605
Test	0.844 (0.798–0.884)	81.74 (78.01–85.06)	59.62 (45.44–72.73)	84.42 (80.84–87.65)	31.63	94.53	40.38	0.413

95% confidence intervals (CIs) for the area under the receiver operating characteristic curve were computed using the DeLong method; 95%CIs for accuracy, sensitivity, and specificity were computed using the Wilson score interval method. ACC: Accuracy; AUC: Area under the receiver operating characteristic curve; CI: Confidence interval; F1: F1-score; FNR: False-negative rate; NPV: Negative predictive value; PPV: Positive predictive value; SEN: Sensitivity; SPE: Specificity; Test: Independent test cohort; Train: Training cohort; Val: Internal validation cohort.

Table 5 Performance metrics of the ResNeXt50 model for chemotherapy response prediction in the training, internal validation, and independent test cohorts

Group	AUC (95%CI)	ACC (%) (95%CI)	SEN (%) (95%CI)	SPE (%) (95%CI)	PPV (%)	NPV (%)	FNR (%)	F1
Train	0.971	93.34	83.51	95.27	77.52	96.73	16.49	0.804
Val	0.896	88.81	75.79	91.36	63.16	95.07	24.21	0.689
Test	0.848 (0.798–0.890)	82.16 (78.63–85.48)	65.38 (51.06–77.78)	84.19 (80.96–87.30)	33.33	95.26	34.62	0.442

95% confidence intervals (CIs) for the area under the receiver operating characteristic curve were computed using the DeLong method; 95%CIs for accuracy, sensitivity, and specificity were computed using the Wilson score interval method. ACC: Accuracy; AUC: Area under the receiver operating characteristic curve; CI: Confidence interval; F1: F1-score; FNR: False negative rate; NPV: Negative predictive value; PPV: Positive predictive value; SEN: Sensitivity; SPE: Specificity; Test: Independent test cohort; Train: Training cohort; Val: Internal validation cohort.

Table 6 Performance metrics of the carbohydrate antigen 19-9-based random forest model for chemotherapy response prediction in the training, internal validation, and independent test cohorts

Group	AUC (95%CI)	ACC (%) (95%CI)	SEN (%) (95%CI)	SPE (%) (95%CI)	PPV (%)	NPV (%)	FNR (%)	F1
Train	1.000	100.00	100.00	100.00	100.00	100.00	0	1.000
Val	0.517	76.97	9.09	95.80	37.50	79.17	90.91	0.146
Test	0.673 (0.445–0.877)	78.95 (65.79–92.11)	25.00 (0.00–60.00)	93.33 (83.33–100.00)	50.00	82.35	75.00	0.333

ACC: Accuracy; AUC: Area under the receiver operating characteristic curve; CI: Confidence interval; F1: F1-score; FNR: False negative rate; NPV: Negative predictive value; PPV: Positive predictive value; SEN: Sensitivity; SPE: Specificity; Train: Training cohort; Test: Independent test cohort; Val: Internal validation cohort.