Artificial intelligence in gastroenterology and hepatology: Status and challenges

Table 3 Summary of key studies on artificial intelligence-assisted pathology in the gastroenterology and hepatology fields

Ref.	Country	Disease studied	Design of study	Application	Number of cases	Type of machine learning algorithm	Outcomes (%)
							Accuracy	Sensitivity/Specificity
Basic AI-based pathology: diagnosis
Tomita et al[118], 2019	United States	BE and EAC	Retrospective	Detection and classification of cancerous and precancerous esophagus tissue	Training: 379 images with 4 classes: normal, BE-no-dysplasia, BE-with-dysplasia, and adenocarcinoma; Testing: 123 images with 4 classes: normal, BE-no-dysplasia, BE-with-dysplasia, and adenocarcinoma	CNNs	Mean: 83; BE-no-dysplasia: 85; BE-with-dysplasia: 89; Adenocarcinoma: 88	Normal: 69/71 BE-no-dysplasia: 77/88; BE-with-dysplasia: 21/97; Adenocarcinoma: 71/91
Sharma et al[119], 2017	Germany	GC	Retrospective	Classification and necrosis detection of GC	454 patients (6810 WSIs: 4994 for cancer classification and 1816 for necrosis detection) (HER2 immunohistochemical stain and HE stained)	CNNs	Cancer classification: 69.90; Necrosis detection: 81.44	NA/NA
Li et al[120], 2018	China	GC	Retrospective	Detection of GC	700 images: 560 GC and 140 normal (HE stained)	CNNs	100	NA/NA
Leon et al[121], 2019	Colombia	GC	Retrospective	Detection of GC	40 images: 20 benign and 20 malignant	CNNs	89.72	NA/NA
Sun et al[122], 2019	China	GC	Retrospective	Diagnosis of GC	500 WSIs of gastric areas with typical cancerous regions	DNNs	91.6	NA/NA
Ma et al[123], 2020	China	GC	Retrospective	Classification of lesions in the gastric mucosa	Training: 534 WSIs (1616713 images: 544925 normal, 544624 chronic gastritis, and 527164 cancer) (HE stained) Testing: 153 WSIs (399240 images: 135446 normal, 125783 chronic gastritis, and 138011 cancer) (HE stained)	CNNs, RF	Benign and cancer: 98.4; Normal, chronic gastritis, and GC: 94.5	Benign and cancer: 98.0/98.9; Normal, chronic gastritis, and GC: NA/NA
Yoshida et al[124], 2018	Japan	Gastric lesions	Retrospective	Classification of gastric biopsy specimens	3062 gastric biopsy specimens (HE stained)	CNNs	55.6	89.5/50.7
Qu et al[125], 2018	Japan	Gastric lesions	Retrospective	Classification of gastric pathology images	Training: 1080 patches: 540 benign and 540 malignant; Testing: 5400 patches: 2700 benign and 2700 malignant	CNNs	96.5	NA/NA
Iizuka et al[126], 2020	Japan	Gastric and colonic epithelial tumors	Retrospective	Classification of gastric and colonic epithelial tumors	4128 cases of human gastric epithelial lesions and 4036 of colonic epithelial lesions (HE stained)	CNNs, RNNs	Gastric adenocarcinoma: 97; Gastric adenoma: 99; Colonic adenocarcinoma: 96; Colonic adenoma: 99	NA/NA
Korbar et al[127], 2017	United States	Colorectal polyps	Retrospective	Classification of different types of colorectal polyps on WSIs	Training: 458 WSIs; Testing: 239 WSIs	A modified version of a residual network	93	88.3/NA
Wei et al[128], 2020	United States	Colorectal polyps	Retrospective	Classification of colorectal polyps on WSIs	Training: 326 slides with colorectal polyps: 37 tubular, 30 tubulovillous or villous, 111 hyperplastic, 140 sessile serrated, and 8 normal; Testing: 238 slides with colorectal polyps: 95 tubular, 78 tubulovillous or villous, 41 hyperplastic, and 24 sessile serrated	CNNs	Tubular: 84.5; Tubulovillous or villous: 89.5; Hyperplastic: 85.3; Sessile serrated: 88.7	Tubular: 73.7/91.6; Tubulovillous or villous: 97.6/87.8; Hyperplastic: 60.3/97.5; Sessile serrated: 79.2/89.7
Shapcott et al[129], 2018	UnitedKingdom	CRC	Retrospective	Diagnosis of CRC	853 hand-marked images	CNNs	84	NA/NA
Geessink et al[130], 2019	Netherlands	CRC	Retrospective	Quantification of intratumoral stroma in CRC	129 patients with CRC	CNNs	94.6	91.1/99.4
Song et al[131], 2020	China	CRC	Retrospective	Diagnosis of CRC	Training: 177 slides: 156 adenoma and 21 non-neoplasm; Testing: 362 slides: 167 adenoma and 195 non-neoplasm	CNNs	90.4	89.3/79.0
Wang et al[132], 2015	China	Hepatic fibrosis	Retrospective	Assessment of HBV-related liver fibrosis and detection of liver cirrhosis	Training: 105 HBV patients; Testing: 70 HBV patients	SVM	82	NA/NA
Forlano et al[133], 2020	UnitedKingdom	MAFLD	Retrospective	Detection and quantification of histological features of MAFLD	Training: 100 MAFLD patients; Testing: 146 MAFLD patients	K-means	Steatosis: 97; Inflammation: 96; Ballooning: 94; Fibrosis: 92	NA/NA
Li et al[134], 2017	China	HCC	Retrospective	Nuclei grading of HCC	4017 HCC nuclei patches	CNNs	96.7	G1: 94.3/97.5; G2: 96.0/97.0;G3: 97.1/96.6; G4: 99.5/95.8
Kiani et al[135], 2020	United States	Liver cancer (HCC and CC)	Retrospective	Histopathologic classification of liver cancer	Training: 70 WSIs: 35 HCC and 35 CC Testing: 80 WSIs: 40 HCC and 40 CC	SVM	84.2	72/95
Advanced AI-based pathology: prediction of gene mutations and prognosis
Steinbuss et al[136], 2020	Germany	Gastritis	Retrospective	Identification of gastritis subtypes	Training: 92 patients (825 images: 398 low inflammation, 305 severe inflammation, and 122 A gastritis) (HE stained) Testing: 22 patients (209 images: 122 low inflammation, 38 severe inflammation, and 49 A gastritis) (HE stained)	CNNs	84	A gastritis: 88/89; B gastritis: 100/93; C gastritis: 83/100
Liu et al[137], 2020	China	Gastrointestinal neuroendocrine tumor	Retrospective	Prediction of Ki-67 positive cells	12 patients (18762 images: 5900 positive cells, 6086 positive cells, and 6776 background from ROIs) (HE and IHC stained)	CNNs	97.8	97.8/NA
Kather et al[138], 2019	Germany	GC and CRC	Retrospective	Prediction of MSI in GC and CRC	Training: 360 patients (93408 tiles); Testing: 378 patients (896530 tiles)	CNNs	84	NA/NA
Bychkov et al[139], 2018	Finland	CRC	Retrospective	Prediction of CRC outcome	420 CRC tumor tissue microarray samples	CNNs, RNNs	69	NA/NA
Kather et al[140], 2019	Germany	CRC	Retrospective	Prediction of survival from CRC histology slides	Training: 86 CRC tissue slides (> 100000 HE image patches); Testing: 25 CRC patients (7180 images)	CNNs	98.7	NA/NA
Echle et al[141], 2020	Germany	CRC	Retrospective	Detection of dMMR or MSI in CRC	Training: 5500 patients; Testing: 906 patients	A modified shufflenet DL system	92	98/52
Skrede et al[142], 2020	3R23 Song 2020	CRC	Retrospective	Prediction of CRC outcome after resection	Training: 828 patients (> 12000000 image tiles); Testing: 920 patients	CNNs	76	52/78
Sirinukunwattana et al[143], 2020	UnitedKingdom	CRC	Retrospective	Identification of consensus molecular subtypes of CRC	Training: 278 patients with CRC; Testing: 574 patients with CRC: 144 biopsies and 430 TCGA	Neural networks with domain-adversarial learning	Biopsies: 85; TCGA: 84	NA/NA
Jang et al[144], 2020	South Korea	CRC	Retrospective	Prediction of gene mutations in CRC	Training: 629 WSIs with CRC (HE stained) Testing: 142 WSIs with CRC (HE stained)	CNNs	64.8-88.0	NA/NA
Chaudhary et al[145], 2018	United States	HCC	Retrospective	Identification of survival subgroups of HCC	Training: 360 HCC patients’ data using RNA-seq, miRNA-seq and methylation data from TCGA; Testing: 684 HCC patients’ data (LIRI-JP cohort: 230; NCI cohort: 221; Chinese cohort: 166, E-TABM-36 cohort: 40, and Hawaiian cohort: 27)	DL	LIRI-JP cohort: 75; NCI cohort: 67; Chinese cohort: 69; E-TABM-36 cohort: 77; Hawaiian cohort: 82	NA/NA
Saillard et al[146], 2020	France	HCC	Retrospective	Prediction of the survival of HCC patients treated by surgical resection	Training: 206 HCC (390 WSIs); Testing: 328 HCC (342 WSIs)	CNNs (SCHMOWDER and CHOWDER)	SCHMOWDER: 78; CHOWDER: 75	NA/NA
Chen et al[11], 2020	China	HCC	Retrospective	Classification and gene mutation prediction of HCC	Training: 472 WSIs: 383 HCC and 89 normal liver tissue; Testing: 101 WSIs: 67 HCC and 34 normal liver tissue	CNNs	Classification: 96.0; Tumor differentiation: 89.6; Gene mutation: 71-89	NA/NA
Fu et al[147], 2020	UnitedKingdom	EAC, GC, CRC, and liver cancers	Retrospective	Prediction of mutations, tumor composition and prognosis	17335 HE-stained images of 28 cancer types	CNNs	Variable across tumors/gene alterations	NA/NA

AI: Artificial intelligence; BE: Barrett’s esophagus; EAC: Esophageal adenocarcinoma; CNN: Convolutional neural network; GC: Gastric cancer; WSI: Whole-slide image; NA: Not available; DNN: Deep neural network; RF: Random forests; RNN: Recurrent neural network; CRC: Colorectal cancer; HBV: Hepatitis-B virus; SVM: Support vector machine; MAFLD: Metabolic associated fatty liver disease; HCC: Hepatocellular carcinoma; CC: Cholangiocarcinoma; ROI: Region of interest; IHC: Immunohistochemistry; MSI: Microsatellite instability; dMMR: Mismatch-repair deficiency; TCGA: The Cancer Genome Atlas; DL: Deep learning.