Developing and validating a computed tomography radiomics strategy to predict lymph node metastasis in pancreatic cancer

Abstract

BACKGROUND

Lymph node metastasis (LNM) is a key prognostic factor in pancreatic cancer (PC). Accurate preoperative prediction of LNM remains challenging. Radiomics offers a noninvasive method to extract quantitative imaging features that may aid in predicting LNM.

AIM

To investigate the potential value of a computed tomography (CT)-based radiomics model in prediction of LNM in PC.

METHODS

A retrospective analysis was performed on 168 pathologically confirmed PC patients who underwent contrast-enhanced-CT. Among them, 107 cases had no LNM, while 61 cases had confirmed LNM. These patients were randomly divided into a training cohort (n = 135) and a validation cohort (n = 33). A total of 792 radiomics features were extracted, comprising 396 features from the arterial phase and another 396 from the portal venous phase. The Minimum Redundancy Maximum Relevance and Least Absolute Shrinkage and Selection Operator methods were used for feature selection and Radiomics model construction. The receiver operating characteristic curve was employed to assess the diagnostic potential of the model, and DeLong test was used to compare the area under the curve (AUC) values of the model.

RESULTS

Six radiomics features from the arterial phase and nine from the portal venous phase were selected. The Radscore model demonstrated strong predictive performance for LNM in both the training and test cohorts, with AUC values ranging from 0.86 to 0.94, sensitivity between 66.7% and 91.7%, specificity from 71.4% to 100.0%, accuracy between 78.8% and 91.1%, PPV ranging from 64.7% to 100.0%, and negative predictive value between 84.0% and 93.8%. No significant differences in AUC values were observed between the arterial and portal venous phases in either the training or test set.

CONCLUSION

The preoperative CT-based radiomics model exhibited robust predictive capability for identifying LNM in PC.

Key Words: Computed tomography; Radiomics; Lymph node metastasis; Pancreatic cancer; Model construction

Core Tip: A preoperative computed tomography-based radiomics model demonstrates high accuracy in predicting lymph node metastasis (LNM) in pancreatic cancer, providing a non-invasive tool to guide personalized treatment. Unlike traditional imaging, radiomics detects microstructural patterns invisible to the human eye, enhancing LNM detection irrespective of phase (arterial vs portal). Clinically, this model could refine preoperative staging, identify candidates for curative surgery, or prioritize neoadjuvant chemotherapy for high-risk patients, optimizing outcomes. Prospective validation is needed for broader adoption.

INTRODUCTION

Pancreatic cancer (PC) stands as a formidable challenge in the landscape of oncology, renowned for its aggressive nature, late diagnosis, and dismal prognosis[1,2]. Among the various factors influencing the course of the disease, lymph node metastasis (LNM) plays a pivotal role in determining treatment strategies, prognosis, and patient outcomes[3]. LNM occurs when cancer cells from the primary tumor migrate to nearby lymph nodes via the lymphatic system. In PC, lymph node involvement is frequent and has a significant impact on both disease staging and prognosis. The presence of LNM is indicative of disease progression and often signifies a more advanced stage of PC. The staging of PC, as defined by the Tumor, Node, Metastasis classification system, heavily relies on the presence or absence of LNM. Patients with lymph node involvement are typically categorized as having advanced-stage disease (e.g., stage III or IV), indicating a poorer prognosis compared to those with localized disease. The extent of LNM, such as the number and size of involved nodes, further influences prognostication. Previous studies have shown that patients with no LNM report 5-year survival rates of up to 40%, whereas those with LNM have a survival rate of less than 10%[4,5].

Detection of LNM profoundly influences treatment decisions in patients with PC. Patients with localized disease and no lymph node involvement may be candidates for curative-intent surgical resection, which offers the best chance of long-term survival[6]. However, the presence of LNM may necessitate neoadjuvant chemotherapy (NAC) or radiation therapy to downstage the disease, or palliative interventions to alleviate symptoms and improve quality of life[7,8].

LNM serves as a predictive marker for disease recurrence in patients with PC. Even after surgical resection of the primary tumor, the presence of metastatic spread to regional lymph nodes increases the risk of locoregional recurrence and distant metastasis. Close surveillance and NAC may be recommended for patients with LNM to monitor recurrence and improve survival outcomes. Previous reports have indicated that PC patients with potentially resectable cancers who underwent NAC followed by curative surgery exhibited improved survival and longer time to recurrence[6-8]. This is particularly notable for those with LNM, underscoring the importance of a pre-treatment diagnosis of LNM as a critical determinant for developing a more personalized treatment strategy in PC patients[9].

Computed tomography (CT) imaging emerges as a primary tool for assessing LNM, offering detailed anatomical visualization and the ability to identify suspicious lymph nodes based on size, morphology, and enhancement characteristics[10]. Advancements in CT technology and imaging protocols have further enhanced its diagnostic accuracy and clinical utility in LNM detection. Multidetector CT scanners, contrast-enhanced (CE) imaging techniques, and functional imaging parameters enable improved visualization and characterization of lymphatic spread, facilitating more precise staging and treatment planning. Additionally, the integration of artificial intelligence and machine learning algorithms holds great promise for enhancing the sensitivity and specificity of CT-based lymph node detection algorithms.

In recent years, the field of radiomics has emerged as a promising approach for extracting quantitative data from medical images to aid in diagnosis, prognosis, and treatment planning[11,12]. Radiomics enables the analysis of tumor characteristics at a much finer level than what is visible to the human eye, allowing for the identification of subtle patterns, textures, and biomarkers that may be indicative of tumor behavior, response to treatment, or overall patient outcomes[13]. The integration of radiomics into clinical practice has the potential to revolutionize cancer care by providing non-invasive, quantitative insights into tumor biology and behavior, leading to more informed decision-making and improved patient outcomes[14]. In this paper, we investigated the potential value of a preoperative CT-based radiomics model in predicting LNM in PC.

MATERIALS AND METHODS

Patients

This study was approved by the institutional review board and patient informed consent was waived due to its retrospective nature. All eligible patients were consecutively recruited from the author’s hospital between January 2019 and January 2022 and informed consent was waived due to its retrospective nature. Inclusion criteria were as follows: (1) Histopathologically confirmed diagnosis of PC; (2) Preoperative imaging indicated resectable disease, excluding those with American Joint Committee on Cancer stage T4[15]; (3) Had ≥ 15 Lymph nodes harvested during surgery to ensure adequate pathological evaluation[16]; and (4) Underwent CE-CT within 30 days prior to surgery. Exclusion criteria were as follows: (1) Absence of CE-CT within 30 days prior to surgery; (2) Incomplete clinical data, histopathological results, or undefined tumor staging; (3) Poor CT image quality unsuitable for tumor segmentation; and (4) Received NAC or radiotherapy before CT imaging. Finally, a total of 168 patients with PC were enrolled in this study, as illustrated in Figure 1. The cases were divided into training and test cohorts (n = 135 and n = 33, respectively) using an 8:2 stratified random sampling method.

Figure 1 Flowchart of patient selection for the study. PC: Pancreatic cancer; CE-CT: Contrast-enhanced computed tomography; LN: Lymph node.

CT examination

All patients underwent a triple-phase CT scan, which included the precontrast, late arterial, and portal venous phases. The CT scanning was conducted using one of the following scanners: (1) GE Optima 670 (GE Healthcare, Tokyo, Japan); (2) GE LightSpeed VCT 64 (GE HealthCare, Milwaukee, Wisconsin, United States); and (3) Philips Brilliance 64 (Philips Healthcare, DA Best, the Netherlands). The scan parameters included a tube voltage of 120 Kvp, tube current ranging from 200 to 400 mAs, helical pitch between 0.984 and 1.375, and a reconstruction slice thickness of 1.0 mm with an interval of 1.0 mm. An administration of 100-120 mL of nonionic contrast media (Omnipaque 350, Bayer Pharmaceuticals) was performed at a rate of 3.0-4.0 mL/second following the precontrast CT scan. The late arterial and portal venous phases were acquired at 35 seconds and 70 seconds, respectively.

Tumor segmentation

Three-dimensional regions of interest (ROIs) were manually delineated on thin-slice CT images during the late arterial and portal venous phases using ITK-SNAP software (version 3.6.0; http://www.itksnap.org/pmwiki/pmwiki.php). Radiologist 1, who has 9 years of abdominal imaging experience and 6 years of tumor segmentation experience involving 221 patients with confirmed pancreatic diseases[11,17], performed the initial ROI delineations along the tumor margins. To assess reproducibility, the intraclass correlation coefficient (ICC) was calculated for intra-observer agreement. Thirty patients were randomly selected, and their ROIs were redelineated by radiologist 1 and by a second radiologist, who has 5 years of abdominal imaging experience and 2 years of tumor segmentation experience, after a 4-week interval. Both radiologists were blinded to the LNM status to reduce bias. An ICC value of 0.75 or higher was considered indicative of good agreement and retained for further analysis.

Feature extraction and selection

All radiomics features were normalized using z-score normalization prior to feature extraction and selection, which helps reduce variability due to differences in scanner parameters and acquisition protocols. The ICC was used to assess the repeatability of each radiomics feature, both intra- and inter-observer. In our study, only the features with ICC values greater than 0.75 were included. To reduce dimensionality and remove redundant or irrelevant features, we employed a two-stage selection approach, first applying minimum redundancy maximum relevance (MRMR), followed by least absolute shrinkage and selection operator (LASSO). Initially, MRMR was performed to eliminate redundant and irrelevant features, resulting in the retention of 30 features. Subsequently, LASSO was conducted to select the optimized subset of features to construct the final model.

Construction and validation of the model

The Radscore was calculated by applying linear weighting to the selected features identified by the LASSO algorithm. The diagnostic efficacy of the model for predicting LNM was assessed using receiver operating characteristic (ROC) curve analysis. The area under the ROC curve (AUC) with 95%CI, along with specificity, sensitivity, accuracy, negative predictive value (NPV), and positive predictive value (PPV), were utilized to evaluate the diagnostic performance of the radiomics models. Additionally, decision curve analysis was employed to assess the radiomics model by calculating the net benefit at different threshold probabilities.

Statistical analysis

Statistical analysis was conducted using SPSS v.24 (IBM Corp., Chicago, IL, United States) and R software v.3.6.1. Non-normally distributed data were compared between groups using the Mann-Whitney U test, while normally distributed continuous data were analyzed using Student's t-test after confirming normality. Normal data were presented as mean ± SD, and categorical data were reported as counts and percentages. The χ² test was applied to compare categorical variables between groups. Inter-observer reproducibility of radiomics features was assessed using the ICC, with coefficients greater than 0.75 indicating good reproducibility. The diagnostic performance of the radiomics model was evaluated using ROC curve analysis. The DeLong test was employed to compare the AUC values of the Radscore model in predicting LNM in PC. A significance level of P < 0.05 was considered statistically significant.

RESULTS

Patient and tumor characteristics

A total of 168 PC patients were ultimately enrolled in this study. The characteristics of PC patients with and without LNM are summarized in Table 1. No significant differences were observed between the two groups in terms of age, gender, clinical symptoms, or tumor markers.

Table 1 Demographic characteristics of pancreatic cancer patients with or without lymph node metastasis, n (%)/mean ± SD/median (25^th-75^th percentiles).

Variables	PC with LNM (n = 61)	PC without LNM (n = 107)	P value
Age (years)	63.65 ± 10.95	61.72 ± 10.66	0.2141
Gender			0.5012
Male	43 (70.5)	70 (65.4)
Female	18 (29.5)	37 (34.6)
Clinical symptoms
Abdominal pain	45 (73.8)	69 (64.5)	0.2152
Abdominal bloating or diarrhea	12 (19.7)	19 (17.8)	0.7582
Yellow urine or icterus	18 (29.5)	30(28.0)	0.8392
Marasmus	9 (14.8)	14 (13.1)	0.7622
Asymptomatic	22 (36.1)	31 (29.0)	0.3412
Tumor markers
CA125	22.80 (15.60-39.40)	20.50 (3.20-34.70)	0.3253
CA199	205.75 (43.90-954.8)	178.96 (39.98-832.67)	0.1053
CEA	4.59 (2.68-6.02)	3.99 (2.44-5.80)	0.4323
AFP	2.95 (2.11-4.54)	2.65 (1.85-4.04)	0.6653

¹Calculated with a t-test.

²Calculated with a Fisher’s exact test or an χ² test.

³Calculated with Mann-Whitney U test.

LNM: Lymph node metastasis; PC: Pancreatic cancer; CEA: Carcinoembryonic antigen; AFP: Alpha-fetoprotein.

Feature extraction and selection

The Analysis Kit software (version V3.0.0.R, GE Healthcare) was utilized to extract a total of 792 radiomics features from each patient's ROIs, with 396 features obtained from the late arterial phase and another 396 from the portal venous phase. These radiomics features, extracted from each phase of CT imaging, encompassed various categories, including 42 histogram features, 9 morphological features, 144 grey level co-occurrence matrix features, 11 grey level size zone matrix features, 180 grey level run-length matrix features, and 10 Haralick features. The MRMR and LASSO algorithms were employed to identify the most informative subset of features from the original set. As a result, 6 features from the late arterial phase and 9 features from the portal venous phase were selected for subsequent analysis, as shown in Figure 2A and B. Following the determination of the number of features using MRMR and LASSO methods, we selected the most predictive subset of features and evaluated the corresponding coefficients, as depicted in Figure 2C and D.

Figure 2 Radiomics feature selection and coefficient visualization using LASSO regression. A and B: The optimal regularization parameter (λ) for the LASSO regression model was selected via 10-fold cross-validation in the training (A) and test (B) cohorts. The plots display binomial deviance against log(λ), with the vertical dashed line indicating the optimal log(λ) of -4. This resulted in 6 non-zero features for the training cohort and 9 for the test cohort; C and D: The corresponding coefficients of the selected radiomics features are shown for the training (C) and test (D) cohorts. The Y-axis lists the selected features, and the X-axis shows their respective coefficients.

Development and validation of CT-based radiomics model

The Radscore was calculated by summing the selected features weighted by their coefficients. For the late arterial phase: Radscore = 0.653 * original_shape_Flatness + 0.132 * original_shape_Sphericity + -0.256 * original_firstorder_10Percentile + -0.411 * original_firstorder_Median + 0.164 * original_glszm_LargeAreaHighGrayLevelEmphasis + 1.217 * original_ngtdm_Coarseness + -0.2. For the portal venous phase: Radscore = 1.18 * original_shape_Flatness + -0.227 * original_shape_Maximum²DDiameterRow + -0.11 * original_shape_MinorAxisLength + 0.079 * original_shape_Sphericity + -0.73 * original_firstorder_Median + 0.068 * original_gldm_SmallDependenceLowGrayLevelEmphasis + 0.53 * original_glszm_SizeZoneNonUniformityNormalized + -0.279 * original_glszm_ZoneEntropy + 1.62 * original_ngtdm_Coarseness + -0.358. Subsequently, we compared the Radscores between PC with and without LNM, in both the training and test cohorts, with separate analyses conducted for the late arterial and portal venous phases, as depicted in Figure 3A and B.

Figure 3 Diagnostic performance of the Radscore model. A and B: Distribution of Radscore values for predicting LNM in the training (A) and test (B) cohorts. Blue dots represent patients without LNM, and yellow dots represent those with LNM; C and D: Receiver operating characteristic curves of the Radscore model for the training (C) and test (D) cohorts. Radscore_A refers to the model derived from arterial phase computed tomography images, while Radscore_P refers to the model derived from portal venous phase images. The area under the curve was used to evaluate model performance.

We then conducted ROC analysis to assess the performance of the model, as presented in Table 2 and illustrated in Figure 3C and D. In the training set, the Radscore model achieved an AUC of 0.92 (0.87-0.97), demonstrating 89.8% sensitivity, 84.9% specificity, 77.2% PPV, 93.6% NPV, and 86.7% accuracy in the arterial phase. Similarly, in the portal venous phase, the Radscore model attained an AUC of 0.94 (0.90-0.98), with 85.7% sensitivity, 94.2% specificity, 89.4% PPV, 92.0% NPV, and 91.1% accuracy in the training set. In the test set, the Radscore model exhibited an AUC of 0.86 (0.73-1.00), with 91.7% sensitivity, 71.4% specificity, 64.7% PPV, 93.8% NPV, and 78.8% accuracy in the arterial phase. Similarly, in the portal venous phase, the Radscore model demonstrated an AUC of 0.93 (0.83-1.00), with 66.7% sensitivity, 100.0% specificity, 100.0% PPV, 84.0% NPV, and 87.9% accuracy. The DeLong test was utilized to compare the AUC values of the Radscore model in predicting LNM. No significant difference was observed in AUC values between the late arterial and portal venous phases in the training set (P = 0.51) or in the test set (P = 0.41). The calibration curves for the training and test cohorts are depicted in Figure 4A and B, respectively. Subsequently, we utilized decision curve analysis to evaluate the clinical utility of the model in both cohorts, as demonstrated in Figure 4C and D.

Figure 4 Calibration and decision curve analysis of the Radscore model. A and B: Calibration curves for the Radscore model in the training (A) and test (B) cohorts. The gray diagonal line represents perfect prediction. The blue and yellow dotted lines show the calibration performance of Radscore_P (portal venous phase) and Radscore_A (arterial phase), respectively; C and D: Decision curve analysis for the Radscore model in the training (C) and test (D) cohorts. The X-axis indicates the threshold probability, and the Y-axis shows the net clinical benefit. The gray and black lines represent treat-all and treat-none strategies, respectively. The radiomics models, particularly Radscore_P, provided higher net benefit than Radscore_A across most threshold ranges in the test cohort (D). AUC: Area under the curve.

Table 2 Diagnostic performance of Radscore model for prediction of lymph node metastasis in pancreatic cancer patients using receiver operating characteristic analysis.

Group	Model	AUC (95%CI)	Accuracy	Sensitivity	Specificity	PPV	NPV
Training set	Radscore_A	0.92 (0.87-0.97)	86.7%	89.8%	84.9%	77.2%	93.6%
	Radscore_P	0.94 (0.90-0.98)	91.1%	85.7%	94.2%	89.4%	92.0%
Test set	Radscore_A	0.86 (0.73-1.00)	78.8%	91.7%	71.4%	64.7%	93.8%
	Radscore_P	0.93 (0.83-1.00)	87.9%	66.7%	100.0%	100.0%	84.0%

AUC: Area under the curve; PPV: Positive predictive value; NPV: Negative predictive value.

DISCUSSION

PC is considered one of the deadliest forms of cancer, primarily due to its tendency to be diagnosed at an advanced stage when treatment options are limited. PC patients with positive LNM experience a poorer prognosis regardless of whether they undergo surgical resection. The presence of LNM serves as a significant prognostic indicator in PC patients, influencing treatment decisions. For PC patients with positive lymph node involvement, considering NAC or immunotherapy prior to surgical resection is advisable[18,19]. Therefore, accurately assessing lymph node status prior to surgical resection is essential in clinical practice. Early detection is particularly critical for tumors measuring ≤ 2 cm, as determining lymph node involvement enables appropriate staging and identification of patients with stage I PC (T1N0M0)[20]. Presently, assessing LNM through imaging examinations lacks both sensitivity and specificity, posing challenges in preoperatively identifying LNM. Consequently, there arises a necessity to develop a predictive model that is both sensitive and efficient for preoperative assessment of LNM in PC patients.

CT is advised as the primary imaging method for assessing LNM in PC[20]. Nevertheless, a meta-analysis revealed that utilizing CT to evaluate extraregional LNM in pancreatic and periampullary cancer resulted in a pooled sensitivity of 25% and a PPV of 28%[21]. Radiomics entails rapidly advancing research focused on extracting quantitative metrics, referred to as radiomic features, from medical images. These features characterize tissue and lesion properties such as heterogeneity and shape and possess the potential, either individually or when combined with demographic, histologic, genomic, or proteomic data, to contribute to clinical decision-making[22]. Due to the significant role of preoperative lymph node status, several investigations have constructed predictive models for LNM by integrating radiomics models with clinical characteristics. These predictive models have been applied across different diseases, including gastric cancer, biliary tract cancer, endometrial cancer, and colorectal cancer[23–26]. In this study, we explored the potential value of preoperative CT-based radiomics model in predicting LNM in PC.

In our study, we utilized the MRMR method to eliminate redundant and irrelevant features, resulting in the retention of 30 features. Following this, we employed the LASSO method to identify the most predictive subset of features and determined their coefficients to calculate the Radscore. Six radiomics features were selected from the late arterial phase, and 9 features were chosen from the portal venous phase for model construction. Our Radscore model exhibited excellent predictive prowess for LNM across both training and test cohorts during the late arterial and portal venous phases, boasting impressive metrics: An AUC ranging from 0.86 (0.73-1.00) to 0.94 (0.90-0.98), sensitivity between 66.7% and 91.7%, specificity from 71.4% to 100.0%, accuracy spanning 78.8% to 91.1%, PPV ranging from 64.7% to 100.0%, and NPV between 84.0% and 93.8%. The Delong test showed that there was no significant difference in AUC values between the late arterial and portal venous phases in the training set (P = 0.51) or in the test set (P = 0.41). Lu et al[27] recently devised a multimodal model that integrates multiphase CE-CT imaging and clinical characteristics, including experts’ experience, to preoperatively predict LNM in 186 PC patients. This model exhibited outstanding predictive performance in diagnosing LNM, achieving an AUC of 0.937, sensitivity of 87.10%, specificity of 87.18%, PPV of 84.38%, NPV of 89.47%, and an accuracy of 93.7% in the training set. In the test set, it maintained high performance with an AUC of 0.923, sensitivity of 80.77%, specificity of 90.00%, PPV of 91.30%, NPV of 78.24%, and an accuracy of 84.78%. Nonetheless, they proposed with 3 kernelled support tensor machine-based classifiers, which incorporates multiphase CE-CT imaging and clinical characteristics alongside experts’ input, presents greater complexity and lacks adaptability in clinical practice. The decision curve analysis showed that the radiomics model provides a greater net benefit than either the treat-all or treat-none strategies across a wide range of threshold probabilities, indicating its potential to guide individualized treatment planning. For example, patients identified as high-risk for LNM based on the model could be considered for more extensive lymph node dissection or neoadjuvant therapy. Nonetheless, further clinical validation and integration with existing clinical workflows are needed to confirm its practical utility.

This study has several limitations. First, it was a retrospective, single-center study, which may introduce selection bias and limit generalizability. Future studies should adopt prospective, multicenter designs to improve robustness and external validity. Second, the model has not been externally validated using independent datasets. Validation with data from other institutions is necessary to confirm its applicability in broader clinical settings. Third, the test cohort size (n = 33) was relatively small, potentially affecting statistical power and model stability. Expanding the test cohort in future research is recommended. Forth, while multiple features were selected, their clinical relevance and biological interpretation require further investigation to enhance model transparency and clinical utility. Finally, a direct comparison between the radiomics model and conventional clinical or radiological assessment methods was not performed, which limits the assessment of incremental value. Future research should include comparative analyses with established clinical models to better evaluate clinical relevance.

CONCLUSION

In conclusion, the radiomics model shows promising performance in identifying LNM in patients with PC. However, its clinical applicability remains limited by concerns about reproducibility and generalizability. Future research should focus on large-scale, multicenter studies with standardized imaging protocols. The use of federated learning may further support model validation and facilitate clinical implementation.

ACKNOWLEDGEMENTS

We thank all authors for their continuous and excellent support with patient data collection, imaging analysis, statistical analysis and valuable suggestions for the article.