Artificial intelligence-assisted esophageal cancer management: Now and future

Table 1 Computer-aided endoscopic diagnosis for dysplastic Barrett’s esophagus

Ref.	Year	Study design	Lesions	Diagnostic method	AI technology	Dataset capacity	Validation	Outcomes	Compared to expert	Processing speed
Münzenmayer et al[53]	2009	Retrospective	BE	WLI	Color-texture analysis in a CBIR framework	390 images with 482 ROIs	LOO (N-fold cross-validation)	Accuracy: BE/CC/EP 70%/74%/95%	NA	NA
van der Sommen et al[55]	2016	Retrospective	HGD, early EAC	WLI	SVM	100 images	LOO	Per-image SEN/SPE: 83%/83%; Per-patient SEN/SPE: 86%/87%	Inferior	NA
Horie et al[56]	2019	Retrospective	EAC	WLI; NBI	CNN-SSD	8 patients	Caffe DL framework	Accuracy: 90%; Per-image SEN: WLI/NBI: 69%/71%; Per-case SEN: WLI/NBI: 88%/88%	NA	0.02 s/image
Ghatwary et al[57]	2019	Retrospective	EAC	WLI	VGG’16-based; R-CNN; Fast R-CNN; Faster R-CNN; SSD	100 images (train 50, test 50)	5-fold cross-validation and LOO	F-measure: 0.94 (SSD); SEN/SPE: 96%/92% (SSD)	NA	0.1-0.2 s/image
Hashimoto et al[58]	2020	Retrospective	HGD, early EAC	WLI and NBI with both standard and near focus	CNN	1835 images	NA	Per-image accuracy: 95.4%; Per-image SEN/SPE: 96.4%/94.2%; 98.6%/88.8% (WLI); 92.4%/99.2% (NBI)	NA	GPU gtx 1070: 0.014 s/frame; YOLO v2: 0.022 s/frame
Ebigbo et al[59]	2019	Retrospective	Early EAC	WLI; NBI	CNN-ResNet	248 images	LOO	SEN/SPE of Augsburg database: 97%/88% (WLI); 94%/80% (NBI); SEN/SPE of MICCAI database: 92%/100%	Superior	NA
de Groof et al[60]	2019	Retrospective	Early dysplastic BE	WLI	ResNet-UNet hybrid	1704 images (train 1544, validation 160)	4-fold cross-validation (external validation)	Accuracy/SEN/SPE: 89%/90%/88% (dataset 4); 88%/93%/83% (dataset 5)	NA (superior to non-expert)	Classification: 0.111 s/image; Segmentation: 0.124 s/image
Swager et al[62]	2017	Retrospective	HGD, early EAC	VLE	SVM, DA, Adaboost, RF, kNN, NB, LR, LogReg	60 images	LOO	AUC: 0.95; SEN/SPE: 90%/93%	Superior	NA
van der Sommen et al[63]	2018	Retrospective	HGD, early EAC	VLE	SVM, RF; AdaBoost; CNN, kNN; DA, LogReg	60 frames	LOO	AUC: 0.90-0.93	Superior	24 ms/full dataset for clinically-inspired features
Struyvenberg et al[65]	2020	Prospective	HGD, early EAC	VLE	PCA-CAD	3060 frames	NA	AUC of Multi-frame: 0.91; AUC of Single-frame: 0.83	NA	0.001 s/frame; 1.5s/full VLE scan
van der Putten et al[66]	2020	Prospective	HGD, early EAC	VLE	Multi-step PDE-CNN on an A-line basis	In-vivo: 140 images (train 111, test 29)	4-fold cross-validation	AUC: 0.93; F1 score: 87.4%	NA	50000 A-lines/s
Shin et al[67]	2016	Retrospective	HGD, EAC	HRM	Two-class LDA-based automated sequential classification algorithm	230 sites (train 77, validation 153)	NA	Accuracy: 84.9%; SEN/SPE: 88%/85%	NA	52 s/image
Qi et al[68]	2006	Retrospective	Dysplastic BE	OCT	PCA	106 images	LOO	Accuracy: 83%; SEN/SPE: 82%/74%	NA	NA

AdaBoost: Adaptive boost; AUC: Area under ROC curve; BE: Barrett’s esophagus; CAD: Computer-aided diagnosis; CBIR: Content-based image retrieval; CC: Mucosa of cardia; CNN: Convolutional neural network; DA: Discriminant analysis; EAC: Esophageal adenocarcinoma; EP: Epithelium; HGD: High-grade dysplasia; HRM: High-resolution microendoscopy; Knn: K-nearest neighbor; LDA: Linear discriminant analysis; LogReg: Logistic regression; LOO: Leave-one-out cross-validation; LR: Linear regression; NA: Not available; NB: Naïve bayes; NBI: Narrow band imaging; OCT: Optical coherence tomography; PCA: Principle component analysis; PDE: Principle dimension encoding; R-CNN: Regional-based CNN; RF: Random forest; SEN: Sensitivity; SPE: Specificity; SSD: Single shot multibox detector; SVM: Support vector machine; VLE: Volumetric laser endomicroscopy; WLI: White light imaging.

Table 2 Computer-aided endoscopic diagnosis for early esophageal squamous cell cancer

Ref.	Year	Study design	Lesions	Diagnostic method	AI technology	Dataset capacity	Validation	Outcomes	Compared to expert	Processing speed
Liu et al[71]	2016	Retrospective	Early ESCC	WLI	JDPCA + CCV	400 images	10-fold cross-validation	Accuracy: 90.75%; AUC: 0.9471; SEN/SPE: 93.33%/89.2%	NA	NA
Horie et al[56]	2019	Retrospective	ESCC	WLI; NBI	CNN-SSD	41 pts (train 8428 images; test 1118 images without histology distinction)	Caffe DL framework	Accuracy: 99%; Per-image SEN: 72%/86% ( WLI/NBI, respectively); Per-case SEN: 79%/89% ( WLI/NBI, respectively)	NA	0.02 s/image
Cai et al[72]	2019	Retrospective	Early ESCC	WLI	DNN	2615 images (train 2428, test 187)	NA	Accuracy: 91.4%; SEN/SPE: 97.8%/85.4%	Superior	NA
Zhao et al[74]	2019	Retrospective	Early ESCC	ME + NBI	Double labeling FNN	1350 images with 1383 lesions	3-fold cross-validation	Accuracy/SEN/SPE at lesion level: 89.2%/87%/84.1%; Accuracy at pixel level: 93%	Comparable	NA
Ohmori et al[73]	2020	Retrospective	Superficial ESCC	ME + NBI/BLI; Non-ME + WLI/NBI/BLI	CNN	23289 images (train 22562, test 727)		Accuracy/SEN/SPE: 77%/100%/63% (Non-ME + NBI/BLI); 81%/90%76% ( Non-ME + WLI); 77%/98%/56% ( ME)	Comparable	0.028 s/image
Nakagawa et al[76]	2019	Retrospective	ESCC (EP-SM1/SM2+SM3)	ME; Non-ME	CNN-SSD	15252 images (train 14338, test 914)	Caffe DL framework	Accuracy/SEN/SPE: 91%/90.1%/95.8%	Comparable	0.033 s/image
Everson et al[77]	2019	Retrospective	ESCC IPCLs (type A/type B)	ME + NBI	CNN	7046 images	5-fold cross-validation+eCAM	Accuracy/SEN/SPE: 93.3%/89.3%/98%	NA	0.026-0.037 s/image
Guo et al[79]	2020	Retrospective	Early ESCC	NBI (ME + non-ME)	CNN-SegNet	13144 images (train 6473, validation 6671), 80 videos (47 lesions, 33 normal esophagus)	NA	Per-image SEN/SPE: 98.04%/95.03%; Per-frame SEN/SPE: 91.5%/99.9%	NA	< 0.04 s/frame; Latency <0.1 s
Shin et al[82]	2015	Retrospective	HGD, ESCC	HRM	Two-class LDA	375 sites of images (train 104, test 104, validation 167)	NA	AUC: 0.95; SEN/SPE: 84%/95%	NA	3.5 s/image
Quang et al[83]	2016	Retrospective	ESCC	HRM	A fully automated algorithm	375 biopsied sites from Shin et al[82] (train 104, test 104, validation 167)	NA	AUC: 0.937; SEN/SPE: 95%/91%	NA	Average 5 s for computing

BLI: Blue laser imaging; CCV: Color coherence vector; DL: Deep learning; CNN: Convolutional neural network; DNN: Deep neural network; eCAM: Explicit class activation map; EP: Epithelium; ESCC: Esophageal squamous cell cancer; FNN: Fuzzy neural network; HGD: High-grade dysplasia; HRM: High-resolution microendoscopy; IPCLs: Intra-papillary capillary loops; JDPCA: Joint diagonalisation principal component analysis; LDA: Linear discriminant analysis; ME: Magnifying endoscopy; NA: Not available; NBI: Narrow band imaging; SEN: Sensitivity; SM: Submucosa; SPE: Specificity; SSD: Single shot multibox detector; WLI: White light imaging.