Artificial intelligence in gastroenterology and hepatology: Status and challenges

Table 1 Summary of key studies on artificial intelligence-assisted endoscopy in gastroenterology fields

Ref.	Country	Disease studied	Design of study	Application	Number of cases	Type of machine learning algorithm	Outcomes (%)
							Accuracy	Sensitivity/Specificity
Esophagogastroduodenoscopy
Takiyama et al[19], 2018	Japan	Anatomical location of upper gastrointestinal tract	Retrospective	Recognition of the anatomical location of upper gastrointestinal tract	Training: 27335 images: 663 larynx, 3252 esophagus, 5479 upper stomach, 7184 middle stomach, 7539 lower stomach, and 3218 duodenum; Testing: 17081 images: 363 larynx, 2142 esophagus, 3532 upper stomach, 6379 middle stomach, 3137 lower stomach, and 1528 duodenum	CNNs	Larynx: 100; Esopha us: 100; Stomach: 99; Duodenum: 99	Larynx: 93.9/100; Esophagus: 95.8/99.7; Stomach: 98.9/93; Duodenum: 87/99.2
Wu et al[20], 2019	China	Diseases of upper gastrointestinal tract	Prospective	Monitor blind spots of upper gastrointestinal tract	Training: 1.28 million images from 1000 object classes; Testing: 3000 images for DCNN1, and 2160 images for DCNN2	CNNs	90.4	87.57/95.02
van der Sommen et al[21], 2016	Netherlands	EN-BE	Retrospective	Detection of EN in BE	21 patients with EN-BE (60 images), 23 patients without EN-BE (40 images)	SVM	NA	86/87
Swager et al[22], 2017	Netherlands	EN-BE	Retrospective	Detection of EN in BE	60 images: 40 with EN-BE and 30 without EN-BE	SVM	95	90/93
Hashimoto et al[23], 2020	United States	EN-BE	Retrospective	Detection of EN in BE	Training: 916 images with EN-BE; Testing: 458 images: 225 dysplasia and 233 non-dysplasia	CNNs	95.4	96.4/94.2
Ebigbo et al[24], 2020	Germany	EAC-BE	Retrospective	Detection of EAC in BE	Training: 129 images; Testing: 62 images: 36 EAC and 26 normal BE	CNNs	89.9	83.7/100
Horie et al[25], 2019	Japan	EAC and ESCC	Retrospective	Detection of EAC and ESCC	Training: 384 patients with 32 EAC and 397 ESCC (8428 images); Testing: 47 patients with 8 EAC and 41 ESCC (1118 images)	CNNs	98	98/79
Kumagai et al[26], 2019	Japan	ESCC	Retrospective	Detection of ESCC	Training: 240 patients (4715 images: 1141 ESCC and 3574 benign lesions); Testing: 55 patients (1520 images: 467 ESCC and 1053 benign)	CNNs	90.9	92.6/89.3
Zhao et al[27], 2019	China	ESCC	Retrospective	Detection of ESCC	165 patients with ESCC and 54 patients without ESCC (1383 images)	CNNs	89.2	87.0/84.1
Cai et al[28], 2019	China	ESCC	Retrospective	Detection of ESCC	Training: 746 patients (2438 images: 1332 abnormal and 1096 normal); Testing: 52 patients (187 images)	CNNs	91.4	97.8/85.4
Nakagawa et al[29], 2019	Japan	ESCC	Retrospective	Determination of invasion depth	Training: 804 patients with ESCC (14338 images: 8660 non-ME and 5678 ME); Testing: 155 patients with ESCC (914 images: 405 non-ME and 509 ME)	CNNs	SM1/SM2, 3: 91.0; Invasion depth: 89.6	SM1/SM2, 3: 90.1/95.8; Invasion depth: 89.8/88.3
Tokai et al[30], 2020	Japan	ESCC	Retrospective	Determination of invasion depth	Training: 1751 images with ESCC; Testing: 42 patients with ESCC (293 images)	CNNs	80.9	84.1/80.9
Ali et al[31], 2018	Pakistan	EGC	Retrospective	Detection of EGC	56 patients with EGC, 120 patients without EGC	SVM	87	91.0/82.0
Sakai et al[32], 2018	Japan	EGC	Retrospective	Detection of EGC	Training: 58 patients (348943 images: 172555 EGC and 176388 normal); Testing: 58 patients (9650 images: 4653 EGC and 4997 normal)	CNNs	87.6	80.0/94.8
Kanesaka et al[33], 2018	Japan	EGC	Retrospective	Detection of EGC	Training: 126 images: 66 EGC and 60 normal; Testing: 81 images: 61 EGC and 20 normal	SVM	96.3	96.7/95.0
Wu et al[34], 2019	China	EGC	Retrospective	Detection of EGC	Training: 9691 images: 3710 EGC and 5981 normal; Testing: 100 patients: 50 EGC and 50 normal	CNNs	92.5	94.0/91.0
Horiuchi et al[35], 2020	Japan	EGC	Retrospective	Detection of EGC	Training: 2570 images: 1492 EGC and 1078 gastritis; Testing: 285 images: 151 EGC and 107 gastritis	CNNs	85.3	95.4/71.0
Zhu et al[36], 2019	China	Invasive GC	Retrospective	Determination of invasion depth	Training: 245 patients with GC and 545 patients without GC (5056 images); Testing: 203 images: 68 GC and 135 normal	CNNs	89.2	76.5/95.6
Luo et al[37], 2019	China	EAC, ESCC, and GC	Prospective	Detection of upper gastrointestinal cancers	Training: 15040 individuals (125898 images: 31633 cancer and 94265 control); Testing: 1886 individuals (15637 images: 3931 cancer and 11706 control)	CNNs	91.5-97.7	94.2/85.8
Nagao et al[38], 2020	Japan	GC	Retrospective	Determination of invasion depth	1084 patients with GC (16557 images); Training: Testing = 4:1	CNNs	94.5	84.4/99.4
Wireless capsule endoscopy
Ayaru et al[39], 2015	United Kingdom	Small bowel bleeding	Retrospective	Prediction of outcomes	Training: 170 patients with small bowel bleeding; Testing: 130 patients with small bowel bleeding	ANNs	Recurrent bleeding 88; Therapeutic intervention: 88; Severe bleeding: 78	Recurrent bleeding: 67/91; Therapeutic intervention: 80/89; Severe bleeding: 73/80
Xiao et al[40], 2016	China	Small bowel bleeding	Retrospective	Detection of bleeding in GI tract	Training: 8200 images: 2050 bleeding and 6150 non-bleeding; Testing: 1800 images: 800 bleeding and 1000 non-bleeding	CNNs	99.6	99.2/99.9
Usman et al[41], 2016	South Korea	Small bowel bleeding	Retrospective	Detection of bleeding in GI tract	Training: 75000 pixels: 25000 bleeding and 50000 non-bleeding; Testing: 8000 pixels: 3000 bleeding and 5000 non-bleeding	SVM	91.8	93.7/90.7
Sengupta et al[42], 2017	United States	Small bowel bleeding	Retrospective	Prediction of 30-d mortality	Training: 4044 patients with small bowel bleeding; Testing: 2060 patients with small bowel bleeding	ANNs	81	87.8/90/9
Leenhardt et al[43], 2019	France	Small bowel bleeding	Retrospective	Detection of GIA	Training: 600 images: 300 hemorrhagic GIA and 300 non-hemorrhagic GIA; Testing: 600 images: 300 hemorrhagic GIA and 300 non-hemorrhagic GIA	CNNs	98	100.0/96.0
Aoki et al[44], 2020	Japan	Small bowel bleeding	Retrospective	Detection of small bowel bleeding	Training: 41 patients (27847 images: 6503 bleeding and 21344 normal); Testing: 25 patients (10208 images: 208 bleeding and 10000 non-bleeding)	CNNs	99.89	96.63/99.96
Yang et al[45], 2020	China	Small bowel polyps	Retrospective	Detection of small bowel polyps	1000 images: 500 polyps and 500 non-polyps	SVM	96.00	95.80/96.20
Vieira et al[46], 2020	Portugal	Small bowel tumors	Retrospective	Detection of small bowel tumors	39 patients (3936 images: 936 tumors and 3000 normal)	SVM	97.6	96.1/98.3
Colonoscopy
Fernández-Esparrach et al[47], 2016	Spain	Colorectal polyps	Retrospective	Detection of polyps	24 videos containing 31 different polyps	Energy maps	79	70.4/72.4
Komeda et al[48], 2017	Japan	Colorectal polyps	Retrospective	Detection of polyps	Training: 1800 images: 1200 adenoma and 600 non-adenoma; Testing: 10 cases	CNNs	70.0	83.3/50.0
Misawa et al[49], 2017	Japan	Colorectal polyps	Retrospective	Detection of polyps	Training: 1661 images: 1213 neoplasm and 448 non-neoplasm; Testing: 173 images: 124 neoplasm and 49 non-neoplasm	SVM	87.8	94.3/71.4
Misawa et al[50], 2018	Japan	Colorectal polyps	Retrospective	Detection of polyps	196631 frames: 63135 polyps and 133496 non-polyps	CNNs	76.5	90.0/63.3
Chen et al[51], 2018	China	Colorectal polyps	Retrospective	Detection of diminutive colorectal polyps	Training: 2157 images: 681 hyperplastic and 1476 adenomas; Testing: 284 images: 96 hyperplastic and 188 adenomas	DNNs	90.1	96.3/78.1
Urban et al[52], 2018	United States	Colorectal polyps	Retrospective	Detection of polyps	Training: 8561 images: 4008 polyps and 4553 non-polyps; Testing: 1330 images: 672 polyps and 658 non-polyps	CNNs	96.4	96.9/95.0
Renner et al[53], 2018	Germany	Colorectal polyps	Retrospective	Differentiation of neoplastic from non-neoplastic polyps	Training: 788 images: 602 adenomas and 186 non-adenomatous polyps; Testing: 186 images: 52 adenomas and 48 hyperplastic lesions	DNNs	78.0	92.3/62.5
Wang et al[54], 2018	United States	Colorectal polyps	Retrospective	Detection of polyps	Training: 5545 images: 3634 polyps and 1911 non-polyps; Testing: 27113 images: 5541 polyps and 21572 non-polyps	CNNs	98	94.4/95.9
Mori et al[55], 2018	Japan	Colorectal polyps	Prospective	A diagnose-and-leave strategy for diminutive, non-neoplastic rectosigmoid polyps	Training: 61925 images; Testing: 466 cases (287 neoplastic polyps, 175 nonneoplastic polyps, and 4 missing specimens)	SVM	96.5	93.8/91.0
Byrne et al[56], 2019	Canada	Colorectal polyps	Retrospective	Detection and classification of polyps	Training: 60089 frames of 223 videos (29% NICE type 1, 53% NICE type 2 and 18% of normal mucosa with no polyp); Testing: 125 videos: 51 hyperplastic polyps and 74 adenoma	CNNs	94.0	98.0/83.0
Blanes-Vidal et al[57], 2019	Denmark	Colorectal polyps	Retrospective	Detection of polyps	131 patients with polyps and 124 patients without polyps	CNNs	96.4	97.1/93.3
Lee et al[58], 2020	South Korea	Colorectal polyps	Retrospective	Detection of polyps	Training: 306 patients (8593 images: 8495 polyp and 98 normal); Testing: 15 patients (15 polyps videos)	CNNs	93.4	89.9/93.7
Gohari et al[59], 2011	Iran	CRC	Retrospective	Determination of prognostic factors of CRC	1219 patients with CRC	ANNs	Colon cancer: 89; Rectum cancer: 82.7	NA/NA
Biglarian et al[60], 2012	Iran	CRC	Retrospective	Prediction of distant metastasis in CRC	1219 patients with CRC	ANNs	82	NA/NA
Takeda et al[61], 2017	Japan	CRC	Retrospective	Diagnosis of invasive CRC	Training: 5543 images: 2506 non-neoplasms, 2667 adenomas, and 370 invasive cancers; Testing: 200 images: 100 adenomas and 100 invasive cancers	SVM	94.1	89.4/98.9
Ito et al[62], 2019	Japan	CRC	Retrospective	Diagnosis of cT1b CRC	Training: 9942 images: 5124 cTis + cT1a, 4818 cT1b, and 2604 cTis + cT1a; Testing: 5022 images: 2604 cTis + cT1a, and 2418 cT1b	CNNs	81.2	67.5/89.0
Zhou et al[63], 2020	China	CRC	Retrospective	Diagnosis of CRC	Training: 3176 patients with CRC and 9003 patients without CRC (464105 images: 28071 CRC and 436034 non-CRC); Testing: 307 patients with CRC and 1956 patients without CRC (84615 images: 11675 CRC and 72940 non-CRC)	CNNs	96.3	91.4/98.0

AI: Artificial intelligence; CNN: Convolutional neural network; EN: Early-stage neoplasia; BE: Barrett’s esophagus; SVM: Support vector machine; NA: Not available; EAC: Esophageal adenocarcinoma; ESCC: Esophageal squamous cell carcinoma; EGC: Early-stage gastric cancer; GC: Gastric cancer; ANN: Artificial neural network; GI: Gastrointestinal; GIA: Gastrointestinal angioectasia; DNN: Deep neural network; CRC: Colorectal cancer.