Advances in the application of artificial intelligence in solid tumor imaging

Abstract

Early diagnosis and timely treatment are crucial in reducing cancer-related mortality. Artificial intelligence (AI) has greatly relieved clinical workloads and changed the current medical workflows. We searched for recent studies, reports and reviews referring to AI and solid tumors; many reviews have summarized AI applications in the diagnosis and treatment of a single tumor type. We herein systematically review the advances of AI application in multiple solid tumors including esophagus, stomach, intestine, breast, thyroid, prostate, lung, liver, cervix, pancreas and kidney with a specific focus on the continual improvement on model performance in imaging practice.

Key Words: Artificial intelligence, Oncology, Imaging, Model performance

Core Tip: Many reviews have summarized artificial intelligence applications in the diagnosis and treatment of a single tumor type. However, this is the first review to systematically review how artificial intelligence relieves clinical workloads and changes the current medical workflows while maintaining high quality to provide precision medicine in multiple solid tumors. Due to its clear advantage in imaging practice, patients will benefit from early diagnosis and appropriate treatment.

INTRODUCTION

Cancer is currently a worldwide health problem. Early diagnosis and timely treatment are crucial in reducing cancer-related mortality. Medical imaging is a common technique used to guide the clinical diagnosis of solid tumors. Accurate interpretation of imaging data has become an important but difficult task in the diagnosis process.

Artificial intelligence (AI) refers to an information science that researches and develops theories, methods, technologies and application systems used to simulate, expand and extend human intelligence[1]. With the rapid development of machine learning, deep learning and other crucial AI technologies in the field of image processing in recent years, these approaches have made great contributions to disease classification, prognosis prediction and therapy evaluation and can identify patterns that humans cannot recognize[2-4] (Figure 1). Here, we review the advantage of AI applications in imaging examinations of multiple solid tumors and highlight its great benefits in optimizing the clinical work process, providing accurate tumor assessment for current precision medicine and achieving better diagnosis and treatment results based on its practical data and literature reports.

Figure 1 A flowchart of artificial intelligence model construction. AI: Artificial intelligence.

APPLICATION OF AI IN GASTROINTESTINAL TUMORS

Gastric cancer is one of the most common gastrointestinal malignancies at present, with a poor prognosis and high mortality. Endoscopy and pathological biopsy are still the “gold standard” for the diagnosis of gastric cancer, but they have shortcomings[5]. For example, the sensitivity of endoscopic diagnosis of atrophic gastritis is only 42%, so the rate of missed diagnosis is relatively high[6]. Multipoint biopsy sampling increases the risk of tissue injury and gastrorrhagia[7,8]. Some advanced endoscopic techniques, such as color endoscopy combined with magnification endoscopy and laser confocal microscopy, can provide only images of the mucosal surface of the gastrointestinal tract[7-9]. Billah et al[10] used capsule endoscopy along with a convolutional neural network (CNN) and color wavelet features to identify gastrointestinal polyps. Urban et al[11] applied deep neural networks to identify colonic polyps from colonoscopy. Lahner et al[12] established a decision support system (DSS) for the diagnosis of atrophic gastritis without endoscopy. The diagnostic accuracy of these three protocols was above 96%, which supports the promising generalization of AI-based technologies.

Esophagus squamous cell cancer

Narrow-band imaging (NBI) is an emerging advanced, noninvasive endoscopic technology that can strengthen the evaluation of the surface structure and microvascular morphology of the esophagus and improve the accuracy rate of endoscopic diagnosis[13]. Using NBI to diagnose squamous cell carcinoma can lead to various results due to different judgments from doctors[14,15]. Fukuda et al[16] applied a deep CNN model to examine NBI endoscopy video images of squamous cell carcinoma, showing higher detection sensitivity (91.1%) than experts and high detection accuracy (88.3%). Those authors suggested that the AI system can discover tumors > 30 mm or with muscularis mucosa invasion that were missed diagnosis by experts. Compared to endoscopic experts, AI has a better diagnostic performance.

Atrophied gastritis

The CNN-chronic atrophic gastritis approach developed by Zhang et al[7] has a good classification performance for recognizing chronic atrophic gastritis based on gastric antrum images whose area under the curve (AUC) was close to 0.99. The accuracy, sensitivity and specificity of CNN-chronic atrophic gastritis in the field of atrophic gastritis diagnosis are all above 0.94. In this study, 1458 mild cases, 1348 moderate cases and 38 severe cases of atrophic gastritis were tested by the CNN model, and the accuracy rates were 0.93, 0.95 and 0.99, respectively, indicating good consistency of the CNN model recognition with the clinical diagnosis of atrophic gastritis.

However, the literature has reported that AI technology used for stomach cancer or esophageal stomach adenocarcinoma is susceptible to problems related to tumor morphology, atrophic change, uneven mucosal background, etc., which leads to low specificity and high false positive rate (FPR)[17]. Several studies indicated that the application of AI in the clinic has high accuracy. If AI technology is combined with endoscopy doctors, then endoscopy can help doctors better diagnose atrophic gastritis, increase the rate of early gastric cancer diagnosis and avoid unnecessary pathological biopsy[18,19].

Early gastric cancer

Regarding small early gastric tumors, Abe et al[18] showed that AI technology can find anomalies faster than endoscopy doctors (45.5 s vs 173.0 min), and it also shows higher sensitivity (58.4% vs 31.9%). However, the positive predictive value (PPV) and specificity of AI technology were relatively lower than those of endoscopy doctors (26.0% vs 46.2% and 87.3% vs 97.0%, respectively)[18]. A computer-aided design (CAD) system is used in stationary images of magnifying endoscopy combined with NBI, which have an accuracy rate for early gastric cancer diagnosis of 85.3%[20]. When endoscopy cannot identify and capture images of lesions, magnifying endoscopy combined with NBI video in the CAD system can help the real-time clinical diagnosis of early gastric cancer. Horiuchi et al[19] proposed that the diagnostic performance of the CAD system using magnifying endoscopy combined with NBI video is equal to or better than that of 11 experienced endoscopic experts in early gastric cancer. The AUC was 0.8684, and its accuracy, sensitivity, specificity, PPV and negative predictive value were 85.1%, 87.4%, 82.8%, 83.5% and 86.7%, respectively[19].

Colorectal cancer

Colorectal colonoscopy is the key technique for the diagnosis of colorectal polyps. However, several studies have shown that 15.4% of colorectal lesions (≤ 3 mm) were diagnosed as adenomas under endoscopy but were judged as normal mucosa via pathological examination[21]. Intraobserver and interobserver discrepancies are the main problem[22]. Therefore, some studies have suggested that using AI techniques combined with endoscopy and imaging may help physicians identify colorectal lesions and perform pathological classification and prognosis prediction[22].

Shahidi et al[21] established a real-time AI-based clinical DSS to assess the differences between results from endoscopy and pathology in lesions ≤ 3 mm. Of the 644 lesions, 458 lesions reached agreement, while significant differences were found in 99 cases (adenoma under endoscopy but normal mucosa by pathologic examination). When using the clinical DSS for further evaluation, they found that the clinical DSS data of 90 cases conformed to those from endoscopy (coincidence rate was 90.9%), supporting AI objectivity prior to pathological examination and interpretation[21]. Yang et al[22] proposed a CNN model whose diagnosis accuracy was better than or similar to that of endoscopic experts (71.5% vs 67.5%), and applications that support the CNN model can help endoscopic physicians identify colorectal lesions to reduce the misdiagnosis rate. The CNN model can also extend the discrimination ability between advanced colorectal cancer and noncancerous lesions, helping endoscopy doctors choose the best treatment strategy effectively[22]. Randomized clinical trials are needed to determine if the CNN model applied to real-time endoscopic video can help endoscopic doctors detect tiny or negligible lesions in the examination.

Wang et al[23] explored the feasibility of faster region-based CNN technology. They used transfer learning technology and images and features of the ImageNet VGG16 model to automatically identify the positive circumferential resection margin in high-resolution magnetic resonance imaging (MRI) of rectal cancer, and the accuracy, sensitivity and specificity were 93.2%, 83.8% and 95.6%, respectively[23]. The use of ¹⁸F fluorodeoxyglucose-positron emission tomography (PET)/computed tomography (CT) to assess early changes in glucose metabolism parameters during neoadjuvant chemotherapy can predict treatment efficacy[24,25]. Traditional ¹⁸F fluorodeoxyglucose-PET/CT cannot accurately and safely select patients for organ preservation strategies[26]. Williams et al[27] suggested that random forest is one type of AI technique used for tumor classification and regression evaluation. Shen et al[28] used random forest to demonstrate that the radiomics obtained from baseline ¹⁸F fluorodeoxyglucose-PET could accurately predict pathological complete response with 95.3% accuracy.

APPLICATION OF AI IN BREAST TUMORS

Ultrasound and radiology are common imaging techniques in breast examination for cancer screening, diagnosis and treatment. Ultrasound is important for the noninvasive measurement of cancer lesions and lymphatic metastasis, increasing the positive diagnostic rate for tiny, aggressive and lymph node-negative breast cancer[29]. However, ultrasound has lower diagnostic specificity and PPV for breast cancer[30]. For example, the axillary positive detection rate of pathological biopsy is 15% to 20%, which is often neglected by ultrasound, especially in those with unspecific characteristics, such as unclear, irregularly shaped edges or fat loss[31]. Although MRI is highly sensitive for the diagnosis of breast cancer, its FPR is as high as 74%[32]. Molybdenum target X-rays are sensitive to microcalcification with the advantage of high cost performance. However, regarding dense breasts where lesions are probably hidden, molybdenum target X-ray has limitations with a lower detection rate[33].

Zhou et al[29] proposed a CNN-based deep learning model to predict lymph node metastasis according to the characteristics of primary breast cancer under ultrasound. The data showed that its AUC was approximately 90%, and the sensitivity and specificity were above 80% and 70%, respectively. Mango et al[30] integrated their AI-based decision support system into ultrasonic images, and the results showed that this technique is helpful in Breast Imaging Reporting and Data System classification, reducing the intraobserver and interobserver variabilities. The variability incidence of ultrasound only in Breast Imaging Reporting and Data System 3 to Breast Imaging Reporting and Data System 4A or above was 13.6%, and it decreased to 10.8% when ultrasound was combined with decision support.

Spick et al[34] showed that adding diffusion-weighted imaging into MRI-guided vacuum-assisted breast biopsy could reduce the FPR by more than 30%. Penco et al[32] verified the accuracy of MRI-guided vacuum-assisted breast biopsy in comparison with histopathological results. The results exhibited 94% accuracy, 84% sensitivity and 77% specificity, with a negative predictive value of up to 97%. Adachi et al[31] compared the diagnostic performance in dynamic contrast-enhanced magnetic resonance for breast cancer detection of AI using RetinaNet to that of expert readers; the former had a higher diagnostic performance than the latter (AUC 0.925 vs 0.884). With the support of AI, the diagnostic performance of expert readers was significantly improved (AUC was 0.899). The sensitivity and specificity of independent AI, experts not using AI and experts using AI in breast cancer diagnosis were 0.926, 0.847, 0.889 and 0.828, 0.841, 0.823, respectively. However, AI may misdiagnose normal breast tissue as malignant due to background parenchymal enhancement or tissue density or misdiagnose invasive ductal carcinoma near the axilla as normal axillary lymph nodes[31].

Sasaki et al[35] proposed that AI-based Transpara systems reduced the differences between computers and experts in the detection sensitivity to breast cancer via molybdenum targets. The expert detection sensitivity was 89%; with the Transpara system, the detection sensitivity for malignant lesions was increased to 95%[35]. When interpreting breast images, the Transpara system can significantly increase AUC and diagnostic sensitivity without increasing reading time[36].

In summary, AI technology increases the detection sensitivity of latent breast lesions while maintaining higher specificity. This technology also reduces the variability in interpretation and helps to improve the clinical diagnostic performance.

APPLICATION OF AI IN THYROID TUMORS

In recent years, with the increasing incidence rate of thyroid cancer, the accurate classification of thyroid lesions and the prediction of lymph node metastasis have been prioritized to be the core of clinical intervention[37,38]. Ultrasound is a noninvasive, easily accessible and economical examination tool, but its accuracy may vary according to the different professional backgrounds of the readers.

Barczyński et al[39] verified that the S-Detect^TM model in real-time CAD system had no significant difference from experienced radiologists in sensitivity, accuracy and negative predictive value of thyroid tumor classification. The overall accuracy of disease evaluation was 76% for surgical doctors who had basic ultrasonic skills not using the CAD system but 82% for doctors with experience using the CAD system[39]. The sensitivity and negative predictive value of lesion classification by the CAD system was similar to those by ultrasonic experts. It further helped to locate the thyroid nodules for further puncture cytology. Nevertheless, the S-Detect^TM model had defects in identifying calcifications[40].

Postoperative lymph node metastasis is a key factor in the local recurrence of thyroid carcinoma. It is necessary to use CT or ultrasound to judge whether lymph node metastasis is present before surgery[37,38]. A study conducted by Lee et al[41] confirmed that the AUC of the CAD system based on deep learning in the classification of thyroid neck lymph node metastasis from preoperative CT images was 0.884, and its diagnostic accuracy, sensitivity, specificity, PPV and negative predictive value were 82.8%, 80.2%, 83.0%, 83.0% and 80.2%, respectively.

APPLICATION OF AI IN PROSTATE CANCER

Serum prostate specific antigen (PSA), digital rectal examination and transrectal prostate ultrasound-guided prostate puncture are the main methods for the early diagnosis of prostate cancer[42]. High-level PSA (> 2 ng/mL) is an important indicator of postoperative monitoring and identifying the recurrence of prostate cancer[43].

Biopsy technology guided by MRI/ultrasound improves the clinical detection of prostate cancer[44,45]. MRI detects pathological changes of Prostate Imaging Reporting and Data System classification is affected by poor intrareader and inter-reader consistency, leading to a 40% difference in targeted biopsy. By adding AI, it will converge Prostate Imaging Reporting and Data System and improve reader consistency, achieving a better (86%) agreement of detected results and pathological diagnosis[46].

Deep learning applications in the field of prostate malignant tumors have been widely used with MRI[47,48]. Although some patients were treated with radical prostate surgery and serum prostate specific antigen < 1, ¹¹C-choline PET/CT still showed a 20.5% positive rate[49]. Prostate uptake of ¹⁸F-choline is associated with the overall survival rate, making it as important as serum prostate specific antigen and Gleason scores in identifying high-risk and low-risk patients. Polymeri et al[50] used an automatic estimation method based on deep learning, and the obtained ¹⁸F-choline uptake value (71 mL) could reach radiologists’ visual estimates (65 mL and 80 mL) within seconds. This approach significantly improved the accuracy and precision of PET/CT imaging in the diagnosis of prostate cancer.

Raciti et al[43] used the software Paige Prostate Alpha to significantly increase the detection rate of prostate cancer while maintaining high specificity. Especially for small, poorly differentiated tumors, the sensitivity can be increased to 30% up to 90%. Similar AI systems can also be used to detect micrometastases in prostate cancer.

APPLICATION OF AI IN LUNG CANCER

When using CT to screen pulmonary nodules, lung-Reporting and Data System can increase sensitivity, but its FPR is also high[51]. The CAD method has 100% sensitivity, but its specificity is extremely low (up to 8.2 false positive nodules per scan)[51]. The negative predictive value of PET/CT for lymph node lesions of peripheral T1 tumors (≤ 3 cm) is as high as 92%-94%[52].

Chauvie et al[51] attempted to apply new methods to digital tomosynthesis: (1) Binomial visual analysis, PPV (0.14) and sensitivity (0.95); (2) Pulmonary-Reporting and Data System, PPV (0.19) and sensitivity (0.65); (3) Logistic regression, PPV (0.29) and sensitivity (0.20); (4) Random forest, PPV (0.40) and sensitivity (0.30); and (5) Neural network, PPV (0.95) and sensitivity (0.90). These data indicated that the neural network was the only predictor of lung cancer with a high PPV value and no loss in sensitivity. Tau et al[52] used CNN to analyze the characteristics of the primary tumor based on PET and to evaluate the existence of lymph node metastasis in newly diagnosed non-small cell lung cancer patients. The sensitivity, specificity and accuracy of predicting positive lymph nodes were 0.74 ± 0.32, 0.84 ± 0.16 and 0.80 ± 0.17, respectively; those of predicting distal metastasis were 0.45 ± 0.08, 0.79 ± 0.06 and 0.63 ± 0.05, respectively. The sensitivity of predicting distant lymph node metastasis was low (24% at prophase and 45% at the end of the monitoring period). CNN had high specificity (91% in the M1 group and 79% in the follow-up group), but the PPV and negative predictive value in class M were lower at the end of follow-up (54.5% and 68.6%).

AI APPLICATION IN OTHER SOLID TUMORS

Hepatocellular carcinoma

The texture analysis of contrast-enhanced magnetic resonance is considered an image tag for predicting the early reaction of hepatocellular carcinoma patients before transarterial chemoembolization (TACE) treatment[53]. Its accuracy for the evaluation of complete remission and incomplete remission was 0.76. Preoperative dynamic CT texture analysis in the prediction of hepatocellular carcinoma response to TACE treatment has certain value. Peng et al[54] used a CT-based deep learning technique (transfer learning) that compensated for the inaccuracy of the result caused by insufficient image information. Further studies showed that the three groups (one training set and two validation sets) of data showed a high AUC for predicting the response to TACE treatment: complete response (0.97, 0.98, 0.97), partial response (0.96, 0.96, 0.96), stable condition (0.95, 0.95, 0.94) and disease progression (0.96, 0.94, 0.97); simultaneously, the accuracy reached 84.0%, 85.1% and 82.8%[54]. Therefore, the CT-based deep learning model helps physicians preliminarily estimate the initial response of hepatocellular carcinoma patients to TACE treatment and helps to predict the therapeutic effect of TACE.

Cervical cancer

Colposcopy is widely used in the detection of cervical intraepithelial neoplasia, and it can guide cervical biopsy in women suspected of having cytological abnormalities or human papillomavirus infection[55,56]. In low- and middle-income countries with a lack of tools for colposcopy, the diagnostic accuracy of cervical biopsy to detect cervical intraepithelial neoplasia is quite low (30%-70%)[57]. The development and application of AI-guided (e.g., support vector machine) digital colposcopy helped solve the bottlenecks and improved the screening effectiveness of cervical cancer to better understand the characteristics of cervical lesions[58]. Another advantage of AI is the “real-time” diagnosis report, which continues to optimize clinical workflows[58].

Pancreatic cancer

Accurate segmentation of the pancreas is important to AI training and AI assisted guidance. Wolz et al[59] used multi atlas technology, which only achieved a dice similarity coefficient (DSC) of 0.70. Summers et al[60] used deep learning technology, which reached a DSC of 0.78%. Wang et al[61] proposed that interactive fully convolutional network for the segmentation of the pancreas did not achieve satisfactory results. Boers et al[62] assumed that the latest interactive U-Net neural structure is better than interactive fully convolutional network because it can produce a better initial segmentation (DSC 78.1% ± 8.7% vs DSC 72.3% ± 11.4%), achieving expert performance faster than artificial division (interactive U-net 8 min to 86% DSC, artificial segmentation 15 min to 87.5% DSC). The average time cost fell 48.4%, but simultaneously due to the low content of visceral fat in some patients, the boundary between the pancreas and surrounding tissues was not clear, which may lead to poor segmentation performance.

Renal cancer

Histopathology is the gold standard for clear cell renal cell carcinoma evaluation[63]. The World Health Organization/International Society of Urological Pathology grading system is used to predict the prognosis of renal clear cell carcinoma[64-66]. Using CT or MRI indications to describe the grading of clear cell renal cell carcinoma is often influenced by subjective factors[67-70]. Cui et al[71] studied the machine learning algorithm to extract and analyze the profiles of tiny tumors. Further grading prediction of clear cell renal cell carcinoma by multiparameter MRI or multiphase CT-based machine learning provides a valuable noninvasive assessment for clinicians in the preoperative treatment of renal tumors[71].

CONCLUSION

AI has clear characteristics of high efficiency, specificity and sensitivity in the classification, identification and diagnosis of solid tumor. After its integration into imaging technology, AI optimizes clinical workflows, decreases the discrepancy between the readers and reduces the misdiagnosis rate, which helps clinicians effectively choose appropriate therapeutic strategies and accurately predict the prognosis (Table 1). All these improvements bring great advantages and convenience to current precision medicine. Nevertheless, problems still exist. For example, the FPR increases due to the morphology of the tumors or the uneven mucosal background and the identification failure of calcification because of technical defects. Therefore, AI cannot be a complete replacement of humans in the contemporary situation. We believe that with the continuous improvement of AI technology, the application of AI in tumor diagnosis and treatment will have better prospects in tumors not limited only to solid tumors.

Table 1 Summary of artificial intelligence application in clinical imaging examination.

Publish date	Ref.	AI	Application scenarios	Sensitivity	Accuracy	Specificity	PPV	NPV	Detection time	Variation	Volume	AUC	DSC
10/2020	Fukuda et al[16]	CNN	Diagnosis of esophagus squamous cell cancer	91.1%	88.3%
05/2020	Zhang et al[7]	CNN	Diagnosis of chronic atrophic gastritis	94.5%	94.2%	94.0%						0.99
10/2020	Horiuchi et al[19]	CAD	Diagnosis of early gastric cancer	87.4%	85.1%	82.8%	83.5%	86.7%				0.8684
02/2020	Wang et al[23]	Faster R-CNN	Circumferential resection margin of rectal cancer	83.8%	93.2%	95.6%
03/2020	Shen et al[28]	RF	Pathological complete response of rectal cancer		95.3%
01/2021	Abe et al[18]	CNN	Diagnosis of gastric cancer	58.4%		87.3%	26.0%		45.5 s
01/2020	Zhou et al[29]	CNN	Lymph node metastasis prediction from primary breast cancer	> 80%		> 70%						0.9
03/2020	Penco et al[32]	DWI	MRI-guided vacuum-assisted breast biopsy	84.0%	94.0%	77.0%		97.0%
05/2020	Adachi et al[31]	RetinaNet	Diagnosis of breast cancer	92.6%		82.8%						0.925
		Readers without RetinaNet		84.7%		84.1%						0.884
		Readers with RetinaNet		88.9%		82.3%						0.899
02/2020	Sasaki et al[35]	Experts	Diagnosis of breast cancer	89.0%
		Experts with Transpara system		95.0%
06/2020	Mango et al[30]	US	Diagnosis of BI-RADS 3 to BI-RADS 4A or above of breast cancer							13.6%
		US+DS								10.8%
02/2020	Barczyński et al[39]	Doctors without CAD	Classification of thyroid tumor		76.0%
		Doctors with CAD			82.0%
06/2020	Lee et al[41]	CAD	Diagnosis of thyroid neck lymph node metastasis	80.2%	82.8%	83.0%	83.0%	80.2%				0.884
03/2020	Polymeri et al[50]	CNN	Prostate gland uptake in PET/CT								71 mL
10/2020	Raciti et al[43]	Paige Prostate Alpha	Diagnosis of prostate cancer	90.0%
07/2020	Chauvie et al[51]	Binomial visual analysis	Lung DTS	95.0%			14.0%
		Pulmonary-RADS		65.0%			19.0%
		Logistic regression		20.0%			29.0%
		RF		30.0%			40.0%
		Neural network		90.0%			95.0%
07/2020	Tau et al[52]	CNN	Diagnosis of lymph node metastasis of lung cancer	74% ± 32%	80% ± 17%	84% ± 16%
			Predicting of distal metastasis of lung cancer	45% ± 8%	63% ± 5%	79% ± 6%	54.5%	68.6%
01/2020	Peng et al[54]	Transfer learning	Predicting of TACE treatment response of hepatocellular carcinoma		> 82.8%							> 0.94
09/2013	Wolz et al[59]	Multi atlas technology	Segmentation of the pancreas										70.0%
08/2020	Gibson et al[62]	Deep learning technology											78.0%
		iFCN											72.3% ± 11.4%
		Artificial segmentation											15 min to 87.5% DSC

AI: Artificial intelligence; PPV: Positive predictive value; NPV: Negative predictive value; AUC: Area under the curve; DSC: Dice similarity coefficient; CNN: Convolutional neural network; Faster R-CNN: Faster region-based convolutional neural network; RF: Random forest; DWI: Diffusion-weighted imaging; US: Ultrasound; DS: Decision support; CAD: Computer-aided design; DTS: Digital tomosynthesis; TACE: Transarterial chemoembolization; iFCN: Interactive fully convolutional network; BI-RADS: Breast Imaging Reporting and Data System; MRI: Magnetic resonance imaging; PET/CT: Positron emission tomography/computed tomography; RADS: Reporting and Data System.