Non-contrast computed tomography radiomics model to predict benign and malignant thyroid nodules with lobe segmentation: A dual-center study

Table 4 Multi-cohort predictive performance of various models

Model	AUC (95%CI)	Accuracy	Sensitivity	Specificity	PPV	NPV	Precision	Recall	F1	Brier
Training cohort (benign = 80, malignant = 184), SMOTE (benign = 104)
LR	0.845 (0.794-0.896)	0.799	0.832	0.725	0.874	0.652	0.874	0.832	0.852	0.140
DT	0.806 (0.745-0.866)	0.826	0.829	0.815	0.946	0.550	0.946	0.829	0.883	0.136
RF	1.000 (1.000-1.000)	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000	0.027
XGB	0.899 (0.845-0.932)	0.845	0.783	0.875	0.935	0.636	0.935	0.783	0.852	0.125
SVM	0.844 (0.791-0.896)	0.814	0.864	0.700	0.869	0.691	0.869	0.864	0.866	0.141
KNN	1.000 (1.000-1.000)	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000	0.031
LGBM	0.692 (0.631-0.752)	0.561	0.755	0.600	0.813	0.516	0.813	0.755	0.783	0.208
Senior radiologist	0.596 (0.505-0.633)	0.542	0.500	0.638	0.760	0.357	0.760	0.500	0.603	0.209
Junior radiologist	0.529 (0.464-0.594)	0.496	0.446	0.613	0.736	0.324	0.726	0.446	0.552	0.211
Validation cohort (benign = 26, malignant = 86)
LR	0.834 (0.750-0.917)	0.768	0.837	0.538	0.857	0.500	0.857	0.837	0.847	0.128
DT	0.646 (0.529-0.762)	0.696	0.802	0.346	0.802	0.346	0.802	0.802	0.802	0.218
RF	0.729 (0.620-0.838)	0.759	0.860	0.423	0.831	0.478	0.831	0.860	0.846	0.166
XGB	0.803 (0.715-0.890)	0.696	0.744	0.538	0.842	0.389	0.842	0.744	0.790	0.144
SVM	0.820 (0.728-0.912)	0.795	0.860	0.577	0.871	0.556	0.871	0.860	0.865	0.130
KNN	0.793 (0.697-0.890)	0.741	0.779	0.615	0.870	0.457	0.870	0.779	0.822	0.180
LGBM	0.724 (0.628-0.820)	0.545	0.430	0.923	0.949	0.329	0.949	0.430	0.592	0.182
Senior radiologist	0.558 (0.449-0.667)	0.527	0.500	0.615	0.811	0.271	0.811	0.500	0.619	0.182
Junior radiologist	0.538 (0.428-0.649)	0.518	0.500	0.577	0.796	0.259	0.796	0.500	0.614	0.182
Temporal test cohort (benign = 26, malignant = 71)
LR	0.814 (0.717-0.912)	0.825	0.930	0.538	0.846	0.737	0.846	0.930	0.886	0.137
DT	0.757 (0.652-0.851)	0.742	0.780	0.533	0.901	0.308	0.901	0.780	0.837	0.178
RF	0.795 (0.696-0.894)	0.763	0.901	0.385	0.800	0.588	0.800	0.901	0.848	0.152
XGB	0.855 (0.775-0.935)	0.773	0.831	0.615	0.855	0.571	0.855	0.831	0.843	0.139
SVM	0.816 (0.719-0.913)	0.845	0.972	0.500	0.841	0.867	0.841	0.972	0.902	0.141
KNN	0.719 (0.607-0.832)	0.711	0.803	0.462	0.803	0.462	0.803	0.803	0.803	0.208
LGBM	0.800 (0.697-0.904)	0.835	0.873	0.731	0.899	0.679	0.899	0.873	0.886	0.193
External test cohort (benign = 15, malignant = 66)
LR	0.782 (0.607-0.907)	0.765	0.848	0.400	0.862	0.375	0.862	0.848	0.855	0.127
DT	0.715 (0.566-0.863)	0.778	0.887	0.421	0.833	0.533	0.833	0.887	0.859	0.166
RF	0.751 (0.589-0.913)	0.815	0.879	0.533	0.892	0.500	0.892	0.879	0.885	0.143
XGB	0.802 (0.644-0.939)	0.728	0.727	0.733	0.923	0.379	0.923	0.727	0.814	0.121
SVM	0.800 (0.674-0.926)	0.815	0.909	0.400	0.870	0.500	0.870	0.909	0.889	0.120
KNN	0.813 (0.697-0.928)	0.803	0.848	0.600	0.903	0.474	0.903	0.848	0.875	0.158
LGBM	0.728 (0.668-0.789)	0.753	0.848	0.333	0.848	0.333	0.848	0.848	0.848	0.163

CI: Confidence interval; SMOTE: Synthetic minority over-sampling technique; DT: Decision trees; KNM: K-nearest neighbors; LR: Logistic regression; RF: Random forests; LGBM: Light gradient boosting machines; SVM: Support vector machines; XGB: Extreme gradient boosting; AUC: Area under the receiver operating characteristic curve; NPV: Negative predictive value; PPV: Positive predictive value.