Artificial intelligence in gastroenterology and hepatology: Status and challenges

Table 1 Summary of key studies on artificial intelligence-assisted endoscopy in gastroenterology fields

Ref.	Country	Disease studied	Design of study	Application	Number of cases	Type of machine learning algorithm	Outcomes (%)
							Accuracy	Sensitivity/Specificity
Esophagogastroduodenoscopy
Takiyama et al[19], 2018	Japan	Anatomical location of upper gastrointestinal tract	Retrospective	Recognition of the anatomical location of upper gastrointestinal tract	Training: 27335 images: 663 larynx, 3252 esophagus, 5479 upper stomach, 7184 middle stomach, 7539 lower stomach, and 3218 duodenum; Testing: 17081 images: 363 larynx, 2142 esophagus, 3532 upper stomach, 6379 middle stomach, 3137 lower stomach, and 1528 duodenum	CNNs	Larynx: 100; Esopha us: 100; Stomach: 99; Duodenum: 99	Larynx: 93.9/100; Esophagus: 95.8/99.7; Stomach: 98.9/93; Duodenum: 87/99.2
Wu et al[20], 2019	China	Diseases of upper gastrointestinal tract	Prospective	Monitor blind spots of upper gastrointestinal tract	Training: 1.28 million images from 1000 object classes; Testing: 3000 images for DCNN1, and 2160 images for DCNN2	CNNs	90.4	87.57/95.02
van der Sommen et al[21], 2016	Netherlands	EN-BE	Retrospective	Detection of EN in BE	21 patients with EN-BE (60 images), 23 patients without EN-BE (40 images)	SVM	NA	86/87
Swager et al[22], 2017	Netherlands	EN-BE	Retrospective	Detection of EN in BE	60 images: 40 with EN-BE and 30 without EN-BE	SVM	95	90/93
Hashimoto et al[23], 2020	United States	EN-BE	Retrospective	Detection of EN in BE	Training: 916 images with EN-BE; Testing: 458 images: 225 dysplasia and 233 non-dysplasia	CNNs	95.4	96.4/94.2
Ebigbo et al[24], 2020	Germany	EAC-BE	Retrospective	Detection of EAC in BE	Training: 129 images; Testing: 62 images: 36 EAC and 26 normal BE	CNNs	89.9	83.7/100
Horie et al[25], 2019	Japan	EAC and ESCC	Retrospective	Detection of EAC and ESCC	Training: 384 patients with 32 EAC and 397 ESCC (8428 images); Testing: 47 patients with 8 EAC and 41 ESCC (1118 images)	CNNs	98	98/79
Kumagai et al[26], 2019	Japan	ESCC	Retrospective	Detection of ESCC	Training: 240 patients (4715 images: 1141 ESCC and 3574 benign lesions); Testing: 55 patients (1520 images: 467 ESCC and 1053 benign)	CNNs	90.9	92.6/89.3
Zhao et al[27], 2019	China	ESCC	Retrospective	Detection of ESCC	165 patients with ESCC and 54 patients without ESCC (1383 images)	CNNs	89.2	87.0/84.1
Cai et al[28], 2019	China	ESCC	Retrospective	Detection of ESCC	Training: 746 patients (2438 images: 1332 abnormal and 1096 normal); Testing: 52 patients (187 images)	CNNs	91.4	97.8/85.4
Nakagawa et al[29], 2019	Japan	ESCC	Retrospective	Determination of invasion depth	Training: 804 patients with ESCC (14338 images: 8660 non-ME and 5678 ME); Testing: 155 patients with ESCC (914 images: 405 non-ME and 509 ME)	CNNs	SM1/SM2, 3: 91.0; Invasion depth: 89.6	SM1/SM2, 3: 90.1/95.8; Invasion depth: 89.8/88.3
Tokai et al[30], 2020	Japan	ESCC	Retrospective	Determination of invasion depth	Training: 1751 images with ESCC; Testing: 42 patients with ESCC (293 images)	CNNs	80.9	84.1/80.9
Ali et al[31], 2018	Pakistan	EGC	Retrospective	Detection of EGC	56 patients with EGC, 120 patients without EGC	SVM	87	91.0/82.0
Sakai et al[32], 2018	Japan	EGC	Retrospective	Detection of EGC	Training: 58 patients (348943 images: 172555 EGC and 176388 normal); Testing: 58 patients (9650 images: 4653 EGC and 4997 normal)	CNNs	87.6	80.0/94.8
Kanesaka et al[33], 2018	Japan	EGC	Retrospective	Detection of EGC	Training: 126 images: 66 EGC and 60 normal; Testing: 81 images: 61 EGC and 20 normal	SVM	96.3	96.7/95.0
Wu et al[34], 2019	China	EGC	Retrospective	Detection of EGC	Training: 9691 images: 3710 EGC and 5981 normal; Testing: 100 patients: 50 EGC and 50 normal	CNNs	92.5	94.0/91.0
Horiuchi et al[35], 2020	Japan	EGC	Retrospective	Detection of EGC	Training: 2570 images: 1492 EGC and 1078 gastritis; Testing: 285 images: 151 EGC and 107 gastritis	CNNs	85.3	95.4/71.0
Zhu et al[36], 2019	China	Invasive GC	Retrospective	Determination of invasion depth	Training: 245 patients with GC and 545 patients without GC (5056 images); Testing: 203 images: 68 GC and 135 normal	CNNs	89.2	76.5/95.6
Luo et al[37], 2019	China	EAC, ESCC, and GC	Prospective	Detection of upper gastrointestinal cancers	Training: 15040 individuals (125898 images: 31633 cancer and 94265 control); Testing: 1886 individuals (15637 images: 3931 cancer and 11706 control)	CNNs	91.5-97.7	94.2/85.8
Nagao et al[38], 2020	Japan	GC	Retrospective	Determination of invasion depth	1084 patients with GC (16557 images); Training: Testing = 4:1	CNNs	94.5	84.4/99.4
Wireless capsule endoscopy
Ayaru et al[39], 2015	United Kingdom	Small bowel bleeding	Retrospective	Prediction of outcomes	Training: 170 patients with small bowel bleeding; Testing: 130 patients with small bowel bleeding	ANNs	Recurrent bleeding 88; Therapeutic intervention: 88; Severe bleeding: 78	Recurrent bleeding: 67/91; Therapeutic intervention: 80/89; Severe bleeding: 73/80
Xiao et al[40], 2016	China	Small bowel bleeding	Retrospective	Detection of bleeding in GI tract	Training: 8200 images: 2050 bleeding and 6150 non-bleeding; Testing: 1800 images: 800 bleeding and 1000 non-bleeding	CNNs	99.6	99.2/99.9
Usman et al[41], 2016	South Korea	Small bowel bleeding	Retrospective	Detection of bleeding in GI tract	Training: 75000 pixels: 25000 bleeding and 50000 non-bleeding; Testing: 8000 pixels: 3000 bleeding and 5000 non-bleeding	SVM	91.8	93.7/90.7
Sengupta et al[42], 2017	United States	Small bowel bleeding	Retrospective	Prediction of 30-d mortality	Training: 4044 patients with small bowel bleeding; Testing: 2060 patients with small bowel bleeding	ANNs	81	87.8/90/9
Leenhardt et al[43], 2019	France	Small bowel bleeding	Retrospective	Detection of GIA	Training: 600 images: 300 hemorrhagic GIA and 300 non-hemorrhagic GIA; Testing: 600 images: 300 hemorrhagic GIA and 300 non-hemorrhagic GIA	CNNs	98	100.0/96.0
Aoki et al[44], 2020	Japan	Small bowel bleeding	Retrospective	Detection of small bowel bleeding	Training: 41 patients (27847 images: 6503 bleeding and 21344 normal); Testing: 25 patients (10208 images: 208 bleeding and 10000 non-bleeding)	CNNs	99.89	96.63/99.96
Yang et al[45], 2020	China	Small bowel polyps	Retrospective	Detection of small bowel polyps	1000 images: 500 polyps and 500 non-polyps	SVM	96.00	95.80/96.20
Vieira et al[46], 2020	Portugal	Small bowel tumors	Retrospective	Detection of small bowel tumors	39 patients (3936 images: 936 tumors and 3000 normal)	SVM	97.6	96.1/98.3
Colonoscopy
Fernández-Esparrach et al[47], 2016	Spain	Colorectal polyps	Retrospective	Detection of polyps	24 videos containing 31 different polyps	Energy maps	79	70.4/72.4
Komeda et al[48], 2017	Japan	Colorectal polyps	Retrospective	Detection of polyps	Training: 1800 images: 1200 adenoma and 600 non-adenoma; Testing: 10 cases	CNNs	70.0	83.3/50.0
Misawa et al[49], 2017	Japan	Colorectal polyps	Retrospective	Detection of polyps	Training: 1661 images: 1213 neoplasm and 448 non-neoplasm; Testing: 173 images: 124 neoplasm and 49 non-neoplasm	SVM	87.8	94.3/71.4
Misawa et al[50], 2018	Japan	Colorectal polyps	Retrospective	Detection of polyps	196631 frames: 63135 polyps and 133496 non-polyps	CNNs	76.5	90.0/63.3
Chen et al[51], 2018	China	Colorectal polyps	Retrospective	Detection of diminutive colorectal polyps	Training: 2157 images: 681 hyperplastic and 1476 adenomas; Testing: 284 images: 96 hyperplastic and 188 adenomas	DNNs	90.1	96.3/78.1
Urban et al[52], 2018	United States	Colorectal polyps	Retrospective	Detection of polyps	Training: 8561 images: 4008 polyps and 4553 non-polyps; Testing: 1330 images: 672 polyps and 658 non-polyps	CNNs	96.4	96.9/95.0
Renner et al[53], 2018	Germany	Colorectal polyps	Retrospective	Differentiation of neoplastic from non-neoplastic polyps	Training: 788 images: 602 adenomas and 186 non-adenomatous polyps; Testing: 186 images: 52 adenomas and 48 hyperplastic lesions	DNNs	78.0	92.3/62.5
Wang et al[54], 2018	United States	Colorectal polyps	Retrospective	Detection of polyps	Training: 5545 images: 3634 polyps and 1911 non-polyps; Testing: 27113 images: 5541 polyps and 21572 non-polyps	CNNs	98	94.4/95.9
Mori et al[55], 2018	Japan	Colorectal polyps	Prospective	A diagnose-and-leave strategy for diminutive, non-neoplastic rectosigmoid polyps	Training: 61925 images; Testing: 466 cases (287 neoplastic polyps, 175 nonneoplastic polyps, and 4 missing specimens)	SVM	96.5	93.8/91.0
Byrne et al[56], 2019	Canada	Colorectal polyps	Retrospective	Detection and classification of polyps	Training: 60089 frames of 223 videos (29% NICE type 1, 53% NICE type 2 and 18% of normal mucosa with no polyp); Testing: 125 videos: 51 hyperplastic polyps and 74 adenoma	CNNs	94.0	98.0/83.0
Blanes-Vidal et al[57], 2019	Denmark	Colorectal polyps	Retrospective	Detection of polyps	131 patients with polyps and 124 patients without polyps	CNNs	96.4	97.1/93.3
Lee et al[58], 2020	South Korea	Colorectal polyps	Retrospective	Detection of polyps	Training: 306 patients (8593 images: 8495 polyp and 98 normal); Testing: 15 patients (15 polyps videos)	CNNs	93.4	89.9/93.7
Gohari et al[59], 2011	Iran	CRC	Retrospective	Determination of prognostic factors of CRC	1219 patients with CRC	ANNs	Colon cancer: 89; Rectum cancer: 82.7	NA/NA
Biglarian et al[60], 2012	Iran	CRC	Retrospective	Prediction of distant metastasis in CRC	1219 patients with CRC	ANNs	82	NA/NA
Takeda et al[61], 2017	Japan	CRC	Retrospective	Diagnosis of invasive CRC	Training: 5543 images: 2506 non-neoplasms, 2667 adenomas, and 370 invasive cancers; Testing: 200 images: 100 adenomas and 100 invasive cancers	SVM	94.1	89.4/98.9
Ito et al[62], 2019	Japan	CRC	Retrospective	Diagnosis of cT1b CRC	Training: 9942 images: 5124 cTis + cT1a, 4818 cT1b, and 2604 cTis + cT1a; Testing: 5022 images: 2604 cTis + cT1a, and 2418 cT1b	CNNs	81.2	67.5/89.0
Zhou et al[63], 2020	China	CRC	Retrospective	Diagnosis of CRC	Training: 3176 patients with CRC and 9003 patients without CRC (464105 images: 28071 CRC and 436034 non-CRC); Testing: 307 patients with CRC and 1956 patients without CRC (84615 images: 11675 CRC and 72940 non-CRC)	CNNs	96.3	91.4/98.0

AI: Artificial intelligence; CNN: Convolutional neural network; EN: Early-stage neoplasia; BE: Barrett’s esophagus; SVM: Support vector machine; NA: Not available; EAC: Esophageal adenocarcinoma; ESCC: Esophageal squamous cell carcinoma; EGC: Early-stage gastric cancer; GC: Gastric cancer; ANN: Artificial neural network; GI: Gastrointestinal; GIA: Gastrointestinal angioectasia; DNN: Deep neural network; CRC: Colorectal cancer.

Table 2 Summary of key studies on artificial intelligence-assisted radiology in hepatology fields

Ref.	Country	Disease studied	Design of study	Application	Number of cases	Type of machine learning algorithm	Outcomes (%)
							Accuracy	Sensitivity/Specificity
Ultrasound-based medical image recognition
Gatos et al[72], 2016	United States	Hepatic fibrosis	Retrospective	Classification of CLD	85 images: 54 healthy and 31 CLD	SVM	87	83.3/89.1
Gatos et al[73], 2017	United States	Hepatic fibrosis	Retrospective	Classification of CLD	124 images: 54 healthy and 70 CLD	SVM	87.3	93.5/81.2
Chen et al[74], 2017	China	Hepatic fibrosis	Retrospective	Classification of the stages of hepatic fibrosis in HBV patients	513 HBV patients with different hepatic fibrosis (119 S0, 164 S1, 88 S2, 72 S3, and 70 S4)	SVM, Naive Bayes, RF, KNN	82.87	92.97/82.50
Li et al[75], 2019	China	Hepatic fibrosis	Prospective	Classification of the stages of hepatic fibrosis in HBV patients	144 HBV patients	Adaptive boosting, decision tree, RF, SVM	85	93.8/76.9
Gatos et al[76], 2019	United States	Hepatic fibrosis	Retrospective	Classification of CLD	88 healthy individuals (88 F0 fibrosis stage images) and 112 CLD patients (112 images: 46 F1, 16 F2, 22 F3, and 28 F4)	CNNs	82.5	NA/NA
Wang et al[77], 2019	China	Hepatic fibrosis	Prospective	Classification of the stages of hepatic fibrosis in HBV patients	Training: 266 HBV patients (1330 images); Testing: 132 HBV patients (660 images)	CNNs	F4: 100; ≥ F3: 99; ≥ F2: 99	F4: 100.0/100.0; ≥ F3: 97.4/95.7; ≥ F2: 100.0/97.7
Kuppili et al[78], 2017	United States	MAFLD	Retrospective	Detection and characterization of FLD	63 patients: 27 healthy and 36 MAFLD	ELM, SVM	ELM: 96.75; SVM: 89.01	NA/NA
Byra et al[79], 2018	Poland	MAFLD	Retrospective	Diagnosis of the amount of fat in the liver	55 severely obese patients	CNNs, SVM	96.3	100/88.2
Biswas et al[80], 2018	United States	MAFLD	Retrospective	Detection and risk stratification of FLD	63 patients: 27 healthy and 36 MAFLD	CNNs, SVM, ELM	CNNs: 100; SVM: 82; ELM: 92	NA/NA
Cao et al[81], 2020	China	MAFLD	Retrospective	Detection and classification of MAFLD	240 patients: 106 healthy, 57 mild MAFLD, 67 moderate MAFLD, and 10 severe MAFLD	CNNs	95.8	NA/NA
Guo et al[82], 2018	China	Liver tumors	Retrospective	Diagnosis of liver tumors	93 patients with liver tumors: 47 malignant lesions (22 HCC, 5 CC, and 10 RCLM), and 46 benign lesions	DNNs	90.41	93.56/86.89
Schmauch et al[83], 2019	France	FLL	Retrospective	Detection and characterization of FLL	Training: 367 patients (367 images); Testing: 177 patients	CNNs	Detection: 93.5; Characterization: 91.6	NA/NA
Yang et al[84], 2020	China	FLL	Retrospective	Detection of FLL	Training: 1815 patients with FLL (18000 images); Testing: 328 patients with FLL (3718 images)	CNNs	84.7	86.5/85.5
CT/MRI-based medical image recognition
Choi et al[85], 2018	South Korea	Hepatic fibrosis	Retrospective	Staging liver fibrosis by using CT images	Training: 7461 patients: 3357 F0, 113 F1, 284 F2, 460 F3, 3247 F4; Testing: 891 patients: 118 F0, 109 F1, 161 F2, 173 F3, 330 F4	CNNs	92.1–95.0	84.6–95.5/89.9–96.6
He et al[86], 2019	United States	Hepatic fibrosis	Retrospective	Staging liver fibrosis by using MRI images	Training: 225 CLD patients; Testing: 84 patients	SVM	81.8	72.2/87.0
Ahmed et al[87], 2020	Egypt	Hepatic fibrosis	Retrospective	Detection and staging of liver fibrosis by using MRI images	37 patients: 15 healthy and 22 CLD	SVM	83.7	81.8/86.6
Hectors et al[88], 2020	United States	Liver fibrosis	Retrospective	Staging liver fibrosis by using MRI images	Training: 178 patients with liver fibrosis; Testing: 54 patients with liver fibrosis	CNNs	F1-F4: 85; F2-F4: 89; F3-F4: 91; F4: 83	F1-F4: 84/90; F2-F4: 87/93; F3-F4: 97/83; F4: 68/94
Vivanti et al[89], 2017	Israel	Liver tumors	Retrospective	Detection and segmentation of new tumors in follow-up by using CT images	246 liver tumors (97 new tumors)	CNNs	86	70/NA
Yasaka et al[90], 2018	Japan	Liver masses	Retrospective	Detection and differentiation of liver masses by using CT images	Training: 460 patients with liver masses (1068 images: 240 Category A, 121 Category B, 320 Category C, 207 Category D, 180 Category E); Testing: 100 images with liver masses: 21 Category A, 9 Category B, 35 Category C, 20 Category D, 15 Category E	CNNs	84	Category A: 71/NA; Category B: 33/NA; Category C: 94/NA; Category D: 90/NA; Category E: 100/NA
Ibragimov et al[91], 2018	United States	Liver diseases requiring SBRT	Retrospective	Prediction of hepatotoxicity after liver SBRT by using CT images	125 patients undergone liver SBRT: 58 liver metastases, 36 HCC, 27 cholangiocarcinoma, and 4 other histopathologies	CNNs	85	NA/NA
Abajian et al[92], 2018	United States	HCC	Retrospective	Prediction of HCC response to TACE by using MRI images	36 HCC patients treated with TACE	RF	78	62.5/82.1
Zhang et al[93], 2018	United States	HCC	Retrospective	Classification of HCC by using MRI images	20 patients with HCC	CNNs	80	NA/NA
Morshid et al[94], 2019	United States	HCC	Retrospective	Prediction of HCC response to TACE by using CT images	105 HCC patients received first-line treatment with TACE	CNNs	74.2	NA/NA
Nayak et al[95], 2019	India	Cirrhosis; HCC	Retrospective	Detection of cirrhosis and HCC by using CT images	40 patients: 14 healthy, 12 cirrhosis, 14 cirrhosis with HCC	SVM	86.9	100/95
Hamm et al[96], 2019	United States	Common hepatic lesions	Retrospective	Classification of common hepatic lesions by using MRI images	Training: 434 patients with common hepatic lesions; Testing: 60 patients with common hepatic lesions	CNNs	92	92/98
Wang et al[97], 2019	United States	Common hepatic lesions	Retrospective	Demonstration of a proof-of-concept interpretable DL system by using MRI images	60 common hepatic lesions patients	CNNs	NA	82.9/NA
Jansen et al[98], 2019	Netherlands	FLL	Retrospective	Classification of FLL by using MRI images	95 patients with FLL (125 benign lesions: 40 adenomas, 29 cysts, and 56 hemangiomas; and 88 malignant lesions: 30 HCC and 58 metastases)	RF	77	Adenoma: 80/78; Cyst: 93/93; Hemangioma: 84/82; HCC: 73/56; Metastasis: 62/77
Mokrane et al[99], 2020	France	HCC	Retrospective	Diagnosis of HCC in patients with cirrhosis by using CT images	Training: 106 patients: 85 HCC and 21 non-HCC; Testing: 36 patients: 23 HCC and 13 non-HCC	SVM, KNN, RF	70	70/54
Shi et al[100], 2020	China	HCC	Retrospective	Detection of HCC from FLL by using CT images	Training: 359 lesions: 155 HCC and 204 non-HCC; Testing: 90 lesions: 39 HCC and 51 non-HCC	CNNs	85.6	74.4/94.1
Alirr et al[101], 2020	Kuwait	Liver tumors	Retrospective	Segmentation of liver tumors	Training: 100 images with liver tumors;Testing: 31 images with liver tumors	CNNs	95.2	NA/NA
Zheng et al[102], 2020	China	Pancreatic cancer	Retrospective	Pancreas segmentation by using MRI images	20 patients with PDAC	CNNs	99.86	NA/NA
Radiomics
Liang et al[103], 2014	China	HCC	Retrospective	Prediction of recurrence for HCC patients who received RFA	83 patients with HCC receiving RFA as first treatment (18 recurrence and 65 non-recurrence)	SVM	82	67/86
Zhou et al[104], 2017	China	HCC	Retrospective	Characterization of HCC	46 patients with HCC: 21 low-grade (Edmondson grades I and II) and 25 high-grade (Edmondson grades III and IV)	Free-form curve-fitting	86.95	76.00/100.00
Abajian et al[105], 2018	United States	HCC	Retrospective	Prediction of response to intra-arterial treatment	36 patients undergone trans-arterial treatment	RF	78	62.5/82.1
Ibragimov et al[91], 2018	United States	Liver tumors	Retrospective	Prediction of hepatobiliary toxicity of SBRT	125 patients undergone liver SBRT: 58 metapatients, 36 HCC, 27 cholangiocarcinoma, and 4 other primary liver tumor histopathologies	CNNs	85	NA/NA
Morshid et al[94], 2019	United States	HCC	Retrospective	Prediction of HCC response to TACE	105 patients with HCC: 11 BCLC stage A, 24 BCLC stage B, 67 BCLC stage C, and 3 BCLC stage D	CNNs	74.2	NA/NA
Ma et al[106], 2019	China	HCC	Retrospective	Prediction of MVI in HCC	Training: 110 patients with HCC: 37 with MVI and 73 without MVI; Testing: 47 patients with HCC: 18 with MVI and 29 without MVI	SVM	76.6	65.6/94.4
Dong et al[107], 2020	China	HCC	Retrospective	Prediction and differentiation of MVI in HCC	Prediction: 322 patients with HCC: 144 with MVI and 178 without MVI; Differentiation: 144 patients with HCC and MVI	RF, mRMR	Prediction: 63.4; Differentiation: 73.0	Prediction: 89.2/48.4; Differentiation: 33.3/80.0
He et al[108], 2020	China	HCC	Prospective	Prediction of MVI in HCC	Training: 101 patients with HCC; Testing: 18 patients with HCC	LASSO	84.4	NA/NA
Schoenberg et al[109], 2020	Germany	HCC	Prospective	Prediction of disease-free survival after HCC resection	Training: 127 patients with HCC; Testing: 53 patients with HCC	RF	78.8	NA/NA
Zhao et al[110], 2020	China	HCC	Retrospective	Prediction of ER of HCC after partial hepatectomy	Training: 78 patients with HCC: 40 with ER and 38 without ER; Testing: 35 patients with HCC: 18 with ER and 17 without ER	LASSO	80.8	80.0/81.6
Liu et al[111], 2020	China	HCC	Retrospective	Prediction of progression-free survival of HCC patients after RFA and SR	RFA: Training: 149 HCC patients undergone RFA Testing: 65 HCC patients undergone RFA; SR: Training: 144 HCC patients undergone SR Testing: 61 HCC patients undergone SR	Cox-CNNs	RFA: 82.0; SR: 86.3	NA/NA
Chen et al[112], 2021	China	HCC	Retrospective	Prediction of HCC response to first TACE by using CT images	Training: 355 patients with HCC; Testing: 118 patients with HCC	LASSO	81	85.2/77.2

AI: Artificial intelligence; CLD: Chronic liver disease; SVM: Support vector machine; HBV: Hepatitis-B virus; RF: Random forests; KNN: K-nearest neighbor; CNN: Convolutional neural network; NA: Not available; MAFLD: Metabolic associated fatty liver disease; FLD: Fatty liver disease; ELM: Extreme learning machine; HCC: Hepatocellular carcinoma; CC: Cholangiocarcinoma; RCLM: Colorectal cancer liver metastases; DNN: deep neural network; FLL: Focal liver lesions; CLD: Chronic liver disease; SBRT: Stereotactic body radiation therapy; TACE: Transarterial chemotherapy; PDAC: Pancreatic ductal adenocarcinoma; RFA: Radiofrequency ablation; BCLC: Barcelona clinic liver cancer staging; MVI: Microvascular invasion; mRMR: Minimum redundancy maximum relevance; LASSO: Least absolute shrinkage and selection operator; ER: Early recurrence; SR: Surgical resection.

Table 3 Summary of key studies on artificial intelligence-assisted pathology in the gastroenterology and hepatology fields

Ref.	Country	Disease studied	Design of study	Application	Number of cases	Type of machine learning algorithm	Outcomes (%)
							Accuracy	Sensitivity/Specificity
Basic AI-based pathology: diagnosis
Tomita et al[118], 2019	United States	BE and EAC	Retrospective	Detection and classification of cancerous and precancerous esophagus tissue	Training: 379 images with 4 classes: normal, BE-no-dysplasia, BE-with-dysplasia, and adenocarcinoma; Testing: 123 images with 4 classes: normal, BE-no-dysplasia, BE-with-dysplasia, and adenocarcinoma	CNNs	Mean: 83; BE-no-dysplasia: 85; BE-with-dysplasia: 89; Adenocarcinoma: 88	Normal: 69/71 BE-no-dysplasia: 77/88; BE-with-dysplasia: 21/97; Adenocarcinoma: 71/91
Sharma et al[119], 2017	Germany	GC	Retrospective	Classification and necrosis detection of GC	454 patients (6810 WSIs: 4994 for cancer classification and 1816 for necrosis detection) (HER2 immunohistochemical stain and HE stained)	CNNs	Cancer classification: 69.90; Necrosis detection: 81.44	NA/NA
Li et al[120], 2018	China	GC	Retrospective	Detection of GC	700 images: 560 GC and 140 normal (HE stained)	CNNs	100	NA/NA
Leon et al[121], 2019	Colombia	GC	Retrospective	Detection of GC	40 images: 20 benign and 20 malignant	CNNs	89.72	NA/NA
Sun et al[122], 2019	China	GC	Retrospective	Diagnosis of GC	500 WSIs of gastric areas with typical cancerous regions	DNNs	91.6	NA/NA
Ma et al[123], 2020	China	GC	Retrospective	Classification of lesions in the gastric mucosa	Training: 534 WSIs (1616713 images: 544925 normal, 544624 chronic gastritis, and 527164 cancer) (HE stained) Testing: 153 WSIs (399240 images: 135446 normal, 125783 chronic gastritis, and 138011 cancer) (HE stained)	CNNs, RF	Benign and cancer: 98.4; Normal, chronic gastritis, and GC: 94.5	Benign and cancer: 98.0/98.9; Normal, chronic gastritis, and GC: NA/NA
Yoshida et al[124], 2018	Japan	Gastric lesions	Retrospective	Classification of gastric biopsy specimens	3062 gastric biopsy specimens (HE stained)	CNNs	55.6	89.5/50.7
Qu et al[125], 2018	Japan	Gastric lesions	Retrospective	Classification of gastric pathology images	Training: 1080 patches: 540 benign and 540 malignant; Testing: 5400 patches: 2700 benign and 2700 malignant	CNNs	96.5	NA/NA
Iizuka et al[126], 2020	Japan	Gastric and colonic epithelial tumors	Retrospective	Classification of gastric and colonic epithelial tumors	4128 cases of human gastric epithelial lesions and 4036 of colonic epithelial lesions (HE stained)	CNNs, RNNs	Gastric adenocarcinoma: 97; Gastric adenoma: 99; Colonic adenocarcinoma: 96; Colonic adenoma: 99	NA/NA
Korbar et al[127], 2017	United States	Colorectal polyps	Retrospective	Classification of different types of colorectal polyps on WSIs	Training: 458 WSIs; Testing: 239 WSIs	A modified version of a residual network	93	88.3/NA
Wei et al[128], 2020	United States	Colorectal polyps	Retrospective	Classification of colorectal polyps on WSIs	Training: 326 slides with colorectal polyps: 37 tubular, 30 tubulovillous or villous, 111 hyperplastic, 140 sessile serrated, and 8 normal; Testing: 238 slides with colorectal polyps: 95 tubular, 78 tubulovillous or villous, 41 hyperplastic, and 24 sessile serrated	CNNs	Tubular: 84.5; Tubulovillous or villous: 89.5; Hyperplastic: 85.3; Sessile serrated: 88.7	Tubular: 73.7/91.6; Tubulovillous or villous: 97.6/87.8; Hyperplastic: 60.3/97.5; Sessile serrated: 79.2/89.7
Shapcott et al[129], 2018	UnitedKingdom	CRC	Retrospective	Diagnosis of CRC	853 hand-marked images	CNNs	84	NA/NA
Geessink et al[130], 2019	Netherlands	CRC	Retrospective	Quantification of intratumoral stroma in CRC	129 patients with CRC	CNNs	94.6	91.1/99.4
Song et al[131], 2020	China	CRC	Retrospective	Diagnosis of CRC	Training: 177 slides: 156 adenoma and 21 non-neoplasm; Testing: 362 slides: 167 adenoma and 195 non-neoplasm	CNNs	90.4	89.3/79.0
Wang et al[132], 2015	China	Hepatic fibrosis	Retrospective	Assessment of HBV-related liver fibrosis and detection of liver cirrhosis	Training: 105 HBV patients; Testing: 70 HBV patients	SVM	82	NA/NA
Forlano et al[133], 2020	UnitedKingdom	MAFLD	Retrospective	Detection and quantification of histological features of MAFLD	Training: 100 MAFLD patients; Testing: 146 MAFLD patients	K-means	Steatosis: 97; Inflammation: 96; Ballooning: 94; Fibrosis: 92	NA/NA
Li et al[134], 2017	China	HCC	Retrospective	Nuclei grading of HCC	4017 HCC nuclei patches	CNNs	96.7	G1: 94.3/97.5; G2: 96.0/97.0;G3: 97.1/96.6; G4: 99.5/95.8
Kiani et al[135], 2020	United States	Liver cancer (HCC and CC)	Retrospective	Histopathologic classification of liver cancer	Training: 70 WSIs: 35 HCC and 35 CC Testing: 80 WSIs: 40 HCC and 40 CC	SVM	84.2	72/95
Advanced AI-based pathology: prediction of gene mutations and prognosis
Steinbuss et al[136], 2020	Germany	Gastritis	Retrospective	Identification of gastritis subtypes	Training: 92 patients (825 images: 398 low inflammation, 305 severe inflammation, and 122 A gastritis) (HE stained) Testing: 22 patients (209 images: 122 low inflammation, 38 severe inflammation, and 49 A gastritis) (HE stained)	CNNs	84	A gastritis: 88/89; B gastritis: 100/93; C gastritis: 83/100
Liu et al[137], 2020	China	Gastrointestinal neuroendocrine tumor	Retrospective	Prediction of Ki-67 positive cells	12 patients (18762 images: 5900 positive cells, 6086 positive cells, and 6776 background from ROIs) (HE and IHC stained)	CNNs	97.8	97.8/NA
Kather et al[138], 2019	Germany	GC and CRC	Retrospective	Prediction of MSI in GC and CRC	Training: 360 patients (93408 tiles); Testing: 378 patients (896530 tiles)	CNNs	84	NA/NA
Bychkov et al[139], 2018	Finland	CRC	Retrospective	Prediction of CRC outcome	420 CRC tumor tissue microarray samples	CNNs, RNNs	69	NA/NA
Kather et al[140], 2019	Germany	CRC	Retrospective	Prediction of survival from CRC histology slides	Training: 86 CRC tissue slides (> 100000 HE image patches); Testing: 25 CRC patients (7180 images)	CNNs	98.7	NA/NA
Echle et al[141], 2020	Germany	CRC	Retrospective	Detection of dMMR or MSI in CRC	Training: 5500 patients; Testing: 906 patients	A modified shufflenet DL system	92	98/52
Skrede et al[142], 2020	3R23 Song 2020	CRC	Retrospective	Prediction of CRC outcome after resection	Training: 828 patients (> 12000000 image tiles); Testing: 920 patients	CNNs	76	52/78
Sirinukunwattana et al[143], 2020	UnitedKingdom	CRC	Retrospective	Identification of consensus molecular subtypes of CRC	Training: 278 patients with CRC; Testing: 574 patients with CRC: 144 biopsies and 430 TCGA	Neural networks with domain-adversarial learning	Biopsies: 85; TCGA: 84	NA/NA
Jang et al[144], 2020	South Korea	CRC	Retrospective	Prediction of gene mutations in CRC	Training: 629 WSIs with CRC (HE stained) Testing: 142 WSIs with CRC (HE stained)	CNNs	64.8-88.0	NA/NA
Chaudhary et al[145], 2018	United States	HCC	Retrospective	Identification of survival subgroups of HCC	Training: 360 HCC patients’ data using RNA-seq, miRNA-seq and methylation data from TCGA; Testing: 684 HCC patients’ data (LIRI-JP cohort: 230; NCI cohort: 221; Chinese cohort: 166, E-TABM-36 cohort: 40, and Hawaiian cohort: 27)	DL	LIRI-JP cohort: 75; NCI cohort: 67; Chinese cohort: 69; E-TABM-36 cohort: 77; Hawaiian cohort: 82	NA/NA
Saillard et al[146], 2020	France	HCC	Retrospective	Prediction of the survival of HCC patients treated by surgical resection	Training: 206 HCC (390 WSIs); Testing: 328 HCC (342 WSIs)	CNNs (SCHMOWDER and CHOWDER)	SCHMOWDER: 78; CHOWDER: 75	NA/NA
Chen et al[11], 2020	China	HCC	Retrospective	Classification and gene mutation prediction of HCC	Training: 472 WSIs: 383 HCC and 89 normal liver tissue; Testing: 101 WSIs: 67 HCC and 34 normal liver tissue	CNNs	Classification: 96.0; Tumor differentiation: 89.6; Gene mutation: 71-89	NA/NA
Fu et al[147], 2020	UnitedKingdom	EAC, GC, CRC, and liver cancers	Retrospective	Prediction of mutations, tumor composition and prognosis	17335 HE-stained images of 28 cancer types	CNNs	Variable across tumors/gene alterations	NA/NA

AI: Artificial intelligence; BE: Barrett’s esophagus; EAC: Esophageal adenocarcinoma; CNN: Convolutional neural network; GC: Gastric cancer; WSI: Whole-slide image; NA: Not available; DNN: Deep neural network; RF: Random forests; RNN: Recurrent neural network; CRC: Colorectal cancer; HBV: Hepatitis-B virus; SVM: Support vector machine; MAFLD: Metabolic associated fatty liver disease; HCC: Hepatocellular carcinoma; CC: Cholangiocarcinoma; ROI: Region of interest; IHC: Immunohistochemistry; MSI: Microsatellite instability; dMMR: Mismatch-repair deficiency; TCGA: The Cancer Genome Atlas; DL: Deep learning.