Application of artificial intelligence to endoscopy on common gastrointestinal benign diseases

Abstract

Artificial intelligence (AI) has been widely involved in every aspect of healthcare in the preclinical stage. In the digestive system, AI has been trained to assist auxiliary examinations including histopathology, endoscopy, ultrasonography, computerized tomography, and magnetic resonance imaging in detection, diagnosis, classification, differentiation, prognosis, and quality control. In the field of endoscopy, the application of AI, such as automatic detection, diagnosis, classification, and invasion depth, in early gastrointestinal (GI) cancers has received wide attention. There is a paucity of studies of AI application on common GI benign diseases based on endoscopy. In the review, we provide an overview of AI applications to endoscopy on common GI benign diseases including in the esophagus, stomach, intestine, and colon. It indicates that AI will gradually become an indispensable part of normal endoscopic detection and diagnosis of common GI benign diseases as clinical data, algorithms, and other related work are constantly repeated and improved.

Key Words: Artificial intelligence, Endoscopy, Common gastrointestinal benign diseases

Core Tip: In endoscopy, the application of artificial intelligence in early gastrointestinal cancer has been widely concerned. We provide a general conclusion of artificial intelligence endoscopy applications in common gastrointestinal benign diseases, such as Barrett’s esophagus, atrophic gastritis, and colonic polyp. Studies indicate high accuracies and efficiencies. Further related work is needed to boost the real application of artificial intelligence in common gastrointestinal benign diseases in the future.

INTRODUCTION

Artificial intelligence (AI) is essentially a process of learning human thinking and transferring human experience based on mathematics and statistics. Iteration of algorithm, rising data, and improving computing power are cores of AI. Machine learning (ML) is a subset of AI[1], and deep learning is a subset of ML to realize ML[2], where multiple algorithms are structured together in complex layers. Artificial neural networks are one of the most common algorithms of AI[3]. Convolutional neural networks (CNNs) are a kind of supervised deep learning algorithm[4]. Its modified format is defined as deep convolutional neural networks[5]. Recognizing images based on artificial neural networks/CNNs promotes AI penetrating in medicine. Computer-aided diagnosis (CAD) systems are designed to interpret medical images using advances of AI from ML to deep learning[6].

In the field of gastroenterology, diseases of the liver, pancreases, and full digestive tract have been involved. Examples include a deep learning model based on computed tomography images to stage liver fibrosis, a deep learning model constructed to differentiate between precancerous lesions and pancreatic cancers, and a deep learning model used in endoscopy to detect early gastrointestinal (GI) cancers. A study covered five kinds of gastric diseases and showed the diagnostic specificity of the CNNs was higher than that of the endoscopists for early gastric cancer and high-grade intraepithelial neoplasia images (91.2% vs 86.7%). The diagnostic accuracy of the CNNs was close to those of the endoscopists for lesion-free, early gastric cancer and high-grade intraepithelial neoplasia, peptic ulcer (PU), advanced gastric cancer (GC), and gastric submucosal tumor images. The CNNs had an image recognition time of 42 s for all the test set images[7]. In this review, the application and research of AI on common GI benign lesions based on endoscopy were concluded.

LITERATURE SEARCH

This review aimed to make a qualitative only review of the application of AI on common GI benign diseases. We searched the PubMed database for articles that were published in the last 5 years using the term combinations of artificial intelligence and common GI benign lesions [Barrett’s esophagus (BE), esophageal varices (EV), atrophic gastritis (AG), PU, gastric polyp, small bowel capsule endoscopy, colonic polyp/adenoma, and inflammatory bowel diseases (IBDs)]. Articles based on radiological images or other samples, review articles, research articles of early or advanced GI cancers or other cancers, and articles only related to either GI benign diseases or AI were excluded. Two authors independently extracted data. Any disagreement was resolved by discussion until consensus was reached or by consulting a third author. Endoscopic-related results were qualitatively concluded in Table 1. The flowchart was presented in Figure 1.

Figure 1 Flow chart of study selection and logic arrangement of review. AG: Atrophic gastritis; AI: Artificial intelligence; BE: Barrett’s esophagus; CA: Colonic adenoma; CP: Colonic polyp; EV: Esophageal varices; GI: Gastrointestinal; GP: Gastric polyp; IBDs: Inflammatory bowel diseases; PU: Peptic ulcer; SB-CE: Small bowel capsule endoscopy.

Table 1 Application of artificial intelligence on common gastrointestinal benign diseases.

Ref.	Aim and disease	Prospective/retrospective	AI method	Endoscopy image	Training dataset	Validation dataset	Result sensitivity	Result specificity	Result accuracy/AUC
Esophageal benign diseases
de Groof et al[12]	Detecting Barrett’sneoplasia	Retrospective	CAD	WLI images	40 images	A leave one out cross validation	92%	95%	85%1
Jisu et al[39]	Distinguishing BE	Retrospective	CNNs	Endomicroscopic images	262 images	Image distortion methods			80.77%1
Ebigbo et al[40]	Distinguishing BE	Retrospective	CNNs (ResNet)	WLI images	129 images	62 images	83.7%	100.0%	89.9%1
Sehgal et al[41]	Detecting dysplasia in BE	Retrospective	ML (decision trees)	Video recordings(AAC)	40 patients with NDBE and DBE		97%	88%	92%1
de Groof et al[14]	Detecting Barrett’sneoplasia	Retrospective	CNN (CAD (ResNet-UNet))	WLI images	494364 images	1704 images (early stage neoplasia in BE and NDBE from 669 patients)	90%	88%	89%1
Dong et al[16]	Screening high risk EV	Retrospective	ML (Random forest)		238 patients	109 patients			Training set (0.84); Validation set (0.82)
Gastric benign diseases
Zhang et al[42]	Diagnosing CAG	Retrospective	CNNs (DenseNet)	WLI images	5470 images	Five-fold cross validation	94.5%	94.0%	94.2%1
Guimarães et al[43]	DiagnosingCAG	Retrospective	CNNs (VGG16)	WLI images	200 images	70 images(ten-fold cross validation)			93%1/0.98
Horiuchi et al[44]	Differentiating CAG	Retrospective	CNNs (GoogLeNet)	ME-NBI images	1078 images	107 images	95.4%	71.0%	85.3%1/0.85
Zhang et al[7]	Diagnosing PU	Retrospective	CNNs (ResNet34)	WLI images	4200 images	228 images	78.9%	88.4%	86.4%1
Lee et al[45]	Differentiating PU	Retrospective	CNNs (ResNet-50/ Inception v3/VGG16 model)	WLI images	200 images	20 images			92.6%1/85.24%1/91.2%1
Namikawa et al[46]	Classifying gastriccancers and ulcers	Retrospective	CNNs (SSD)	WLI/NBI/chromoendoscopy images	373 images	720 images	93.3%	99.0%	93.3 %1
Zhang et al[26]	Detecting GP	Retrospective	CNNs (SSD-GPNet)	WLI images	404 images	50 images			93.92%1
Intestinal benign diseases
Hwang et al[29]	Classifying hemorrhagic and ulcerations	Retrospective	CNNs (VGGNet)	Capsule endoscopy	7556 images	5760 images	Model 1 vs Model 2; 97.61% vs 95.07%	Model 1 vs Model 2; 96.04% vs 98.18%	Model 1 vs Model 2; 96.83%1 vs 96.62%1
Aoki et al[47]	Detecting erosions and ulcerations	Retrospective	CNNs (SSD)	Capsule endoscopy	5360 images	10440 images	88.2%	90.9%	90.8%1/0.958
Aoki et al[48]	Detecting erosions and ulcerations	Retrospective	CNNs (SSD)	Capsule endoscopy		20 videos
Ding et al[49]	Detecting small bowel diseases	Retrospective	CNNs (ResNet)	Capsule endoscopy	158235 images	5000 patients	99.88% per patient99.90% per lesion	100% per patient100 % per lesion
Fan et al[50]	Detecting erosions and ulcerations	Retrospective	CNNs (AlexNet)	Capsule endoscopy	Ulcer 2000; Erosion 2720	Ulcer 500; Erosion 690	Ulcer: 96.80%; Erosion: 93.67%	Ulcer: 94.79%; Erosion: 95.98%	Ulcer: 95.16%1; Erosion: 95.34%1/0.98
Leenhardt et al[51]	Detecting small bowel angiectasia	Retrospective	CNNs	Capsule endoscopy	300 videos with angiectasia	300 videos with angiectasia	100%	96%
Tsuboi et al[52]	Detecting small bowel angiectasia	Retrospective	CNNs (SSD)	Capsule endoscopy	141 patients	28 patients	98.8%	98.4%	0.998
Colonic benign diseases
Lui et al[34]	Detecting missed colonic lesions	Retrospective and prospective	R-FCN (ResNet101)	Endoscopic videos (WLI)	52 videos	Real-time AI detected at least 1 missed adenoma in 14 patients (26.9%) and increased the total number of adenomas detected by 23.6%.
Rodriguez-Diaz et al[53]	Histologically classifying CP	Retrospective	CAD	NBI	745 images +65000 images		96%	84%
Komeda et al[54]	Diagnosing CP	Retrospective	CNNs-CAD	WLI/NBI/ chromoendoscopy images	1200 images	10-fold cross validation			75.1%1
Akbari et al[55]	Classifying CP	Retrospective	FCNs	WLI images	200 images	300 images
Chen et al[56]	Classifying diminutive CP	Retrospective	DCNNs-CAD	NBI images	96 images + 188 images		96.3%	78.1%
Gong et al[57]	Detecting CA	Prospective	DCNNs	WLI images	DCNNs system (n = 355) or unassisted (control) colonoscopy (n = 349)		58 (16%) of 35527 (8%) of 349
Byrne et al[58]	Differentiating adenomatous and hyperplastic polyps	Retrospective	DCNNs	Videos and NBI images	223 polyp videos	40 videos	98%	83%
Mori et al[59]	Identifying diminutive CP	Prospective	CAD	NBI/stained images	791 consecutive patients undergoing colonoscopy and 23 endoscopists				Pathologic prediction rate of 98.1%1
Misawa et al[60]	DetectingCP	Retrospective	CAD	WLI images	105 positive and 306 negative videos	50 positive and 85 negative videos	90.0%	63.3%	76.5%1
Taunk et al[61]	Classifying polyp histology	Retrospective	CAD	pCLE images	125 images	189 images	95%	94%	94%1
Wang et al[62]	Detecting CA	Prospective	CAD	WLI images	484 patients in the CADe group and 478 in the sham group		165 (34%) of 484; 132 (28%) of 478
Tong et al[63]	Differentiating UC, CD, and ITB	Retrospective	CNNs/RF	WLI images	6399 consecutive patients (5128 UC, 875 CD and 396 ITB)		RF (UC 97%, CD 65%, and ITB 68%); CNN (UC 99%, CD 87%, and ITB 52%)	RF (UC 97%, CD 53%, and ITB 76%); CNN (UC 97%, CD 83%, and ITB 81%)	RF (UC 0.97, CD 0.58, and ITB 0.72); CNN (UC 0.98, CD 0.85, and ITB 0.63)
Ozawa et al[36]	Diagnosing UC	Retrospective	CAD	WLI images	26304 images	3981 images			0.86 (Mayo 0); 0.98 (Mayo 0–1)
Stidham et al[37]	Grading the severity of ulcerative colitis	Retrospective	CNNs	WLI images	2465 patients	308 patients	83.0%	96.0%	0.966
Maeda et al[38]	Identifying histologic inflammation associated with UC	Retrospective	CAD	Endocytoscopic images	87 patients	100 patients	74%	97%	91%1

¹Results accuracy. AAC: Acetic acid chromoendoscopy; AI: Artificial intelligence; AUC: Area under the curve; BE: Barrett’s esophagus; CA: Colorectal adenomas; CAD: Computer-aided diagnosis; CAG: Chronic atrophic gastritis; CD: Crohn’s disease; CNN: Convolutional neural network; CP: Colorectal polyp; DBE: Dysplastic Barrett’s esophagus; DCNNs: Deep convolutional neural networks; EV: Esophageal Varices; FCNs: Fully convolutional networks; GP: Gastric polyp; ITB: Intestinal tuberculosis; ME-NBI: Magnifying narrow-band imaging; ML: Machine learning; NBI: Narrow-band imaging; NDBE: Non-dysplastic Barrett’s esophagus; pCLE: Probe-based confocal laser endomicroscopy; PU: Peptic ulcer; RF: Random forest; R-FCNs: Region-based fully connected convolutional neural networks; SSD: Single shot detector; UC: Ulcerative colitis; WLI: White-light imaging.

SEARCH RESULTS

Initially, a total of 555 articles were identified. After manually screening and reading, only research articles related to the application of AI to common GI benign lesions (BE, EV, AG, PU, gastric polyp, small bowel capsule endoscopy, colonic polyp/adenoma, and IBDs) based on different endoscopic images or tissue slides from endoscopic biopsies were included. Finally, 35 studies were tabulated in Table 1. Six studies demonstrated the application of AI on esophageal benign diseases (5 BE and 1 EV). Seven studies were about gastric benign diseases (3 AG, 3 PU, and 1 polyp). Seven studies were about intestinal diseases. Fifteen studies were about colonic benign diseases (11 polyp/adenoma and 4 IBDs).

AI AND ESOPHAGEAL BENIGN DISEASES: BARRETT’S ESOPHAGUS AND ESOPHAGEAL VARICES

BE is a precursor to esophageal adenocarcinoma. Intestinal metaplasia and gastric metaplasia are two pathological subclasses of BE. Intestinal metaplasia can progress to esophageal cancer. The ablation of dysplastic BE will reduce the risk of progression to cancer[8]. Endoscopic surveillance, including white-light imaging (WLI), narrow-band imaging, and chromoendoscopy, is performed to detect dysplasia in BE. Approximately 5% of the esophageal mucosa is found at risk by random biopsies sample[9].

Recently, AI has been applied in some studies of BE. For example, CAD based on deep learning and different algorithms trained by WLI and endomicroscopic images to detect, diagnose, and distinguish BE with achievable results (the accuracy from 80.77% to 92%, specificity from 88% to 100%, and sensitivity from 83.7% to 97%) (Table 1). On pathology, CAD with wide area transepithelial sampling could increase the detection of high-grade dysplasia/esophageal adenocarcinoma (absolute increase: 14.4%)[10]. Deep convolutional neural networks were used in the whole-slide tissue histopathology images-based diagnosis of dysplastic and non-dysplastic BE[11]. Moreover, distinguishing BE adenocarcinoma by AI methods has been studied based on different endoscopic images such as WLI and volumetric laser endomicroscopic images with accuracy from 88% to 92%, specificity from 88% to 93%, and sensitivity from 90% to 95%[12-14].

As another common esophageal benign disease, EV are associated with cirrhosis and portal hypertension, and variceal hemorrhage is a substantial cause of mortality[15]. However, related AI research is limited. A score system based on ML was built on the data of 238 patients with cirrhosis to reliably identify patients with varices that needed treatments and achieved an area under the curve (AUC) from 0.75 to 0.84 in different groups[16]. Another study of the index of spleen volume-to-platelet ratio based on deep learning-measured spleen volume on computed tomography to assess high-risk varices in B-viral compensated cirrhosis had a sensitivity of 69.4% and specificity of 78.5%[17]. There is little research of AI on esophagitis, although it is also a common esophageal disease associated with BE and esophageal cancer.

AI AND GASTRIC BENIGN LESIONS: ATROPHIC GASTRITIS, PEPTIC ULCER, AND POLYP

Gastritis, peptic ulcer, polyp and adenoma, and vascular lesion are common gastric benign diseases. The detection and diagnosis of these lesions account for a large part of daily endoscopic work. If AI can be applied in this field, then the rate of detection and accuracy will be improved. Moreover, the rapid identification of simple lesions can fill the lack of endoscopists and reduce the workload.

Early diagnosis of chronic AG, a precancerous lesion, is important to prevent the occurrence and development of GC. AI-assisted detection and diagnosis has been related to endoscopic images (Table 1), histological images[18,19], and X-ray images[20,21]. The accuracy was from 85.3% to 94.2%, the specificity was from 71% to 94%, and the sensitivity was from 94.5% to 95.4%. Helicobacter pylori infection, as a dominant cause of chronic AG and GC, has also been detected via AI methods based on endoscopic images, such as CNNs (GoogLeNet) and CNNs (ResNet-50 model), which achieved an accuracy up to 93.8% in a considerably short time of less than 200 s[22-24].

A CNN method was constructed to diagnose PU and differentiate GC from PU mainly based on WLI, narrow-band imaging, and chromoendoscopic images with an accuracy from 85.2% to 93.3%, specificity from 88.4% to 99%, and sensitivity from 78.9% to 93.3% (Table 1). In addition, a ML model was built on six parameters, such as age and the presence of PU, to predict recurrent ulcer bleeding within 1 year with an AUC of 0.775 and an accuracy of 84.3%[25].

There were only a few applications of AI on detecting gastric hyperplastic polyps and adenomas. A 93.92% accuracy was achieved when detecting polyps by CNNs (SSD-GPNet) based on WLI images[26]. A CNN method was trained to detect adenomas and showed an AUC of 0.99 based on histopathology whole-slide images[27]. Research and application of AI on gastric benign lesions are limited, although these diseases make up a considerable part of daily work. Some of them are usually prone to severe outcomes and risks despite the relative ease to diagnose. Indeed, the study of AI on this aspect will assist endoscopists to improve early detection rates and bring the opportunity of early treatment to benefit patients.

AI AND INTESTINAL DISEASES: CAPSULE ENDOSCOPY

The application of AI in small bowel diseases has been concentrated on capsule endoscopy. It includes image enhancement using ML algorithms to reduce artifact interference as well as three-dimensional luminal map reconstruction and localization[28]. AI-assisted capsule endoscopy in detecting ulcer, erosion, bleeding, polyps, parasite, diverticulum, and angiectasia with an accuracy more than 90.0%, specificity from 90.9% to 100%, and sensitivity from 88.2% to 100% in a short time (about 6 min) (Table 1). Furthermore, a gradient class activation map was used to visualize and detect lesions by CNNs-VGGNet to improve the classification and localization[29]. In addition, a CNN method based on conventional abdominal radiographs was trained to detect high-grade small bowel obstruction with an AUC of 0.84, a sensitivity of 83.8%, and a specificity of 68.1%[30]. In another study, it achieved an AUC of 0.971, a sensitivity of 91.4%, and a specificity of 91.9% using region-based CNNs[31]. The limited research indicated CNNs could recognize specific images among a large variety with high efficiency and accuracy. The application of AI will relieve the clinical workload as capsule endoscopy reading is a time-consuming process.

AI AND COLONIC BENIGN LESIONS: POLYP, ADENOMA, AND IBDS

A 1.0% increase of adenoma detection rate has been associated with a 3.0% decrease in the risk of interval colorectal cancer[32]. To improve colorectal polyp and adenoma detection, AI has been widely applied in the detection, real-time histological classification, segmentation, localization, and distinguishing of diminutive polyps and adenomas based on different methods trained by videos and images in retrospective or prospective and in multicenter or single center clinical trials (Table 1). Deep learning was also used to automatically classify colorectal polyps on histopathologic slides[33]. For the internal evaluation, the accuracy of the deep CNN method was 93.5%, which was comparable to the pathologists accuracy of 91.4%. On the external test, it achieved an accuracy of 87.0%, which was comparable to the pathologists accuracy of 86.6%. The application of AI in colorectal polyps has gained more concerns and practice, and it is deeper and closer to the clinical use to further increase the detection rate of polyps. For example, real-time AI detected at least one missed adenoma in 14 patients (26.9%) and increased the total number of adenomas detected by 23.6%[34].

AI methods have been trained in grading endoscopic disease severity of patients with ulcerative colitis and in predicting remission in patients with moderate to severe Crohn’s disease[35]. For example, a CNN-CAD system based on GoogLeNet was robustly promising to identify normal mucosa (Mayo 0) and mucosal healing state with an accuracy of 0.86 of Mayo 0 and of 0.98 of Mayo 0-1[36]. Another similar system could differentiate remission (Mayo 0 or 1) from moderate or severe disease (Mayo 2 or 3) with an AUC of 0.966, a specificity of 96.0%, and a sensitivity of 83.0%[37]. A CAD was constructed to identify the presence of histologic inflammation associated with ulcerative colitis using endocytoscopy with an accuracy of 91%, a specificity of 97%, and a sensitivity 74%[38] (Table 1).

FUTURE PERSPECTIVES OF AI APPLICATION ON COMMON GI BENIGN LESIONS

We summarized the application and research of AI on common GI benign diseases. Limited studies are promising as most of the studies showed comparatively high accuracies and efficiencies. As studies of AI application on gastroenterology continue to increase, there are several areas of interest that will hold significant value in the future. First, the technical integration of AI systems will be important to optimize clinical workflow. New AI applications can easily “read in” data from a video input, allowing the systems to use the data for training and real time decision support. Second, AI systems will continue to expand the clinical applications. Some promising studies have demonstrated how AI can improve our performance on clinical tasks such as polyp identification, detection of small bowel bleeding, and endoscopic recognition of Helicobacter pylori infection, etc. More research, especially randomized controlled trials, on how to train and validate up-to-date algorithms will be continued on the present work to find more precise methods and identify new clinical tasks after practice. Third, further research will be needed to describe the most effective training methods for physician practices beginning to adopt AI technology because AI will be an indispensable helper of normal endoscopic detection and diagnosis of common GI benign lesions in the future.

CONCLUSION

Although AI is a relatively new technology, it has the potential to ease the daily workload of radiologists, pathologists, and sonographers. In endoscopy, AI related to early GI cancers and precancerous lesions has garnered more research than common GI benign diseases, despite the latter occupying a large proportion of daily work and being easier to detect and diagnose than early cancers. If models and diagnosing routes based on AI targeted at common GI benign diseases are well developed, then it will bring great benefits to patients and endoscopists, especially in primary hospitals where medical resources are lacking and core work is mainly focused on early diagnosis and treatment of common GI benign diseases. Furthermore, AI methods and technology targeted at common benign diseases will be easier for endoscopists to adopt professional education. More research is needed to overcome the challenges of integrating AI into the detection of common GI benign diseases by endoscopy, but the future is promising.