Artificial intelligence application in diagnostic gastrointestinal endoscopy - Deus ex machina?

Abstract

The close relationship of medicine with technology and the particular interest in this symbiosis in recent years has led to the development of several computed artificial intelligence (AI) systems aimed at various areas of medicine. A number of studies have demonstrated that those systems allow accurate diagnoses with histological precision, thus facilitating decision-making by clinicians in real time. In the field of gastroenterology, AI has been applied in the diagnosis of pathologies of the entire digestive tract and their attached glands, and are increasingly accepted for the detection of colorectal polyps and confirming their histological classification. Studies have shown high accuracy, sensitivity, and specificity in relation to expert endoscopists, and mainly in relation to those with less experience. Other applications that are increasingly studied and with very promising results are the investigation of dysplasia in patients with Barrett's esophagus and the endoscopic and histological assessment of colon inflammation in patients with ulcerative colitis. In some cases AI is thus better than or at least equal to human abilities. However, additional studies are needed to reinforce the existing data, and mainly to determine the applicability of this technology in other indications. This review summarizes the state of the art of AI in gastroenterological pathology.

Key Words: Artificial intelligence, Computer-aided diagnosis, Deep learning, gastrointestinal endoscopy, Colorectal polyps, Dysplasia

Core Tip: The uses of artificial intelligence (AI) in gastroenterology are growing from day to day. Many published studies have demonstrated the potential benefits of using this technology in clinical practice. We review here the most recent studies related to the major hot topics of AI in gastroenterology, namely colorectal polyps, dysplasia in Barrett's esophagus, and inflammation in ulcerative colitis. The machine seems to cooperate with the human, helping him to exceed his abilities and making the future of this symbiosis very promising.

INTRODUCTION

Artificial intelligence (AI) is a concept that involves the fields of science and engineering, and was defined by one of its founders, Alan Turing, as the ability of a computer to achieve human performance in cognitive tasks[1]. Machine Learning (ML) is an AI subfield that allows the identification of patterns that can be used to analyze a particular situation[2]. ML can "learn" and improve with experience gained from provided data sets and then apply that information to similar scenarios in the future[2]. Deep learning (DL) is an AI subfield that uses artificial neural networks that are similar to the human brain, and which the capacity to learn and make decisions on its own[2]. Since 1970, AI has been used in medicine for various purposes, including improving the effectiveness of diagnosis, facilitating therapeutic and disease monitoring, and improving overall patient outcomes[2,3].

Gastroenterology is a medical specialty with great potential for integrating AI technology, as the diagnosis of gastrointestinal conditions relies much on image-based investigations (endoscopy and radiology)[4]. Numerous recent studies have investigated the applicability of AI in such diverse areas of gastroenterology, as in the investigation of dysplasia in Barrett's esophagus (BE), diagnosis of gastroesophageal reflux disease, differentiation of acute and chronic pancreatitis, detection and classification of colorectal polyps, characterization of colonic inflammatory activity in patients with inflammatory bowel disease (IBD), among others[2].

In this review, we aimed to describe the latest uses of AI in gastroenterology and the evidence that allows understanding how far these developments can supplant human abilities. We conducted a search of several platforms (PubMed, MEDLINE and EMBASE), with no time limit for articles that compared the performance of these systems and endoscopists.

AI APPLICATIONS IN DIAGNOSTIC GASTROINTESTINAL ENDOSCOPY

AI for the detection of dysplasia in Barrett’s esophagus

The epidemiology of esophageal cancer has changed over the years. The increasing incidence of esophageal adenocarcinoma since the 1970s, has made it the most common type of esophageal cancer, mainly in the western countries[5,6]. BE is considered a premalignant condition[5,6] with a risk of progression to esophageal adenocarcinoma of up to 0.3% per year[7]. Endoscopic surveillance programs are based on the fact that progression tends to occur gradually from low-grade to high-grade dysplasia and esophageal adenocarcinoma. The aim is to find premalignant and malignant lesions at an early stage[8].

Because of the possible impact on the survival of the patients, there has been a growing interest in the application of AI and ML in patients with BE[9]. Several studies have reported high rates of sensitivity, specificity and accuracy with computer-aided detection (CAD) systems in detecting and delineating dysplastic lesions in patients with BE[10-13]. In comparison with expert endoscopists, the CAD system had an accuracy of 88% vs 73%, sensitivity of 93% vs 72%, and specificity of 83% vs 74%[10]. In addition, the area of delineation of the lesion by the CAD system overlapped that of expert endoscopists (Figure 1)[10].

Figure 1 Delineation of Barrett's esophagus by computer-aided detection system and expert endoscopists. Images on the left are from a computer-aided detection system to identify and delineate dysplastic lesions in Barrett’s esophagus with white-light imaging. Images on the right show the outlines of the same lesions made by expert endoscopists[13]. Citation: de Groof J, van der Sommen F, van der Putten J, Struyvenberg MR, Zinger S, Curvers WL, Pech O, Meining A, Neuhaus H, Bisschops R, Schoon E, With PH, Bergman JJ. The Argos project: The development of a computer-aided detection system to improve detection of Barrett's neoplasia on white-light endoscopy. United European Gastroenterol J 2019; 7: 538-47. Copyright © The Author(s) 2019. Published by SAGE Publications.

Despite the promising results and the data showing that the detection speed for dysplastic lesions was compatible with the real-time use of endoscopic surveillance, this application has only recently been tried in real time. Thus, Ebigbo et al[14] used the CAD system in 14 patients simultaneously with endoscopic examination by BE-expert endoscopists. The performance of the AI system was similar to that of the experienced endoscopists, with a sensitivity of 83.7%, a specificity of 100%, and an accuracy of 89.9%, suggesting its potential to improve endoscopic surveillance of BE, mainly by nonexpert endoscopists.

Currently, there is no evidence that advanced imaging techniques, such as chromoendoscopy, autofluorescence endoscopy, or confocal laser endomicroscopy, offer advantages over high-definition white-light endoscopy (WLE)[7]. Volumetric laser endomicroscopy (VLE) is a recent endoscopic imaging technology that uses optical coherence tomography to produce high-resolution scans of 6-cm segments of the esophagus, with surface and subsurface image depth greater that 3 mm[15,16]. In this way, the real-time diagnosis of surface and subsurface lesions, as well as guiding their endoscopic treatment is possible[15,16]. VLE scans comprise a large amount of visual information with numerous gray-shade images that make interpretation complex and time consuming even by experts[17-19]. CAD systems have been developed that are better at identifying early BE neoplasia in ex vivo VLE images than VLE experts are[17], mainly when multiframe analysis is used[18]. However, further studies are needed to validate in vivo data. Trindade et al[19] also created AI image enhancement software called intelligent real-time image segmentation, that identifies three VLE features previously associated with histologic dysplasia (i.e. hyper-reflective surface, hyporeflective structures and lack of a layered architecture) and displays them as different colors superimposed over the VLE image to facilitate interpretation. Studies are underway to assess the effectiveness of this AI system.

AI in the detection of colorectal polyps

It is now known that colorectal cancer (CRC) surveillance with colonoscopy of both the right and left colon is associated with a reduction in CRC mortality[20-22]. The importance of a meticulous high-quality colonoscopy is a result of the inverse relationship that exists between the adenoma detection rate and the risk of interval CRC or advanced-stage CRC[23]. The high rate (up to 30%) of missed adenomas during screening colonoscopy led to the development of computer-aided detection (CADe) systems, which are programs based on DL, in an attempt to mitigate this problem[24]. The first study[25] on the use of CADe in the detection of colon polyps was published in 2003, and it demonstrated good results. However, only static images were used. Since then, several systems have been developed and used with real-time endoscopy. In recent systematic review and meta-analysis, Hassan et al[26] assessed the relationship between the increased detection of polyps and the main features of the detected lesions. The review included five randomized clinical trials with a total of 4354 patients (2163 in the CADe and 2191 in the control group). The highlight of the results was that the adenoma detection rate in the CADe group was significantly higher (36.6%) than that in the control group (25.2%) regardless of adenoma size, location, and morphology of the polyps. The rates of detection of advanced adenomas were not significantly different. The authors concluded that AI could benefit colorectal cancer surveillance, as the factors that affect the detection of lesions by human observers, such as size and morphology, do not interfere with detection by the AI system. In addition, the study shows that the AI system did not affect the efficiency of colonoscopy, maintaining similar withdrawal time in both groups. The results are supported by those in prospective study by Liu et al[27] in which the adenoma detection rate was higher in the CADe group than in the control group (29.01% vs 20.91%) and a meta-analysis by Barua et al[28] that found an absolute increase of 10.3% in the detection rate of adenomas. The increase was mainly the result of increased detection of nonadvanced diminutive adenomas, serrated adenomas, and hyperplastic polyps[26,27]. In addition to the smaller size, other characteristics of the polyps first detected by the CADe system were isochromia, flat shape, unclear boundary, partly behind colon folds, and at the edge of the visual field[27]. Thus, the features of the polyps seem to contribute to a higher probability of missed lesions, however Lui et al[29] found that 79% of missed lesions can be detected with AI support during the first examination and considered that the main cause of missed lesions was human error, such as less endoscopist experience or the presence of multiple polyps that can increase endoscopist distraction or fatigue. We did not find consistent evidence for the use of advanced imaging techniques in the detection of polyps by CAD systems.

AI in the histological characterization of colorectal polyps

One of the problems of CRC surveillance is the high cost of polypectomy and pathological examination, in particular given the high prevalence of diminutive polyps ≤ 5 mm[30]. It has been proven that a resect and discard strategy for diminutive polyps has an important economic impact, without losing efficacy in CRC surveillance[30]. An optical biopsy of diminutive polyps supported by image-enhanced endoscopic may be appropriate. The Preservation and Incorporation of Valuable endoscopic Innovations (PIVI) group of the American Society for Gastrointestinal Endoscopy (ASGE)[31] recommends that for a “diagnose-and-leave” strategy, endoscopic diagnosis should provide a 90% or higher negative predictive value for adenomatous histology when used with high confidence. For a “resect and discard” strategy for adenomas ≤ 5 mm, it is recommended that an optical biopsy with high confidence, when combined with a histologic assessment of polyps larger than 5 mm, should provide ≥ 90% agreement in the assignment of post polypectomy surveillance intervals compared with decisions based on pathological assessment of all identified polyps.

The optical diagnosis of colorectal polyps has not yet been widely implemented because of the lack of endoscopist experience and the considerable learning curve of advanced imaging modalities[32,33]. To overcome this problem, AI systems such as, computer-aided diagnostic (CADx) that help to predict the histology (i.e. neoplastic or non-neoplastic) of detected polyps (Figure 2) have been developed with the aim of avoiding unnecessary endoscopic biopsies or resection[33-35]. Yang et al[35] created DL models that classify colorectal lesions histologically using white-light colonoscopy images. They found an accuracy of 79.5% in distinguishing neoplastic and non-neoplastic lesions and 87.1% in distinguishing advanced and nonadvanced colorectal lesions. When comparing the performance of endoscopists and CADx for the classification of polyps, they found that CADx performed better than less experienced endoscopists, but not better than experts.

Figure 2 Computer-aided detection system for detecting and classifying colorectal polyps. The top image is a polypoid lesion (delimited by blue marks) identified by a CADe system. The bottom image is a lesion classified as neoplastic by the CADx system with a high degree of confidence (three gray bars on the right side of the image)[33]. Citation: Mori Y, Neumann H, Misawa M, Kudo SE, Bretthauer M. Artificial intelligence in colonoscopy - Now on the market. What's next? J Gastroenterol Hepatol 2021; 36(1): 7-11. Copyright © The Author(s) 2020. Published by Journal of Gastroenterology and Hepatology Foundation and John Wiley & Sons Australia, Ltd.

Combinations of advanced imaging modalities, such as narrow-band imaging (NBI) or endocytoscopy, and CAD systems improved the performance of AI-assisted optical biopsy[34]. Several studies have shown the potential of combining NBI with CADx in the classification of polyps[36-38]. Song et al[37] developed a DL model to predict colorectal polyp histology based on NBI near-focus images. Their study showed a good diagnostic accuracy (81.3%-82.4%) that was comparable to expert endoscopists and was significantly more accurate than trainee endoscopists. CADx performance was consistent regardless of polyp size, location, or morphology, with an average histological assessment time of 0.02-0.04 s. They concluded that an AI CAD system can help inexperienced endoscopists in the histological assessment of colorectal polyps and increase the confidence of experts regarding their assessments.

A recent systematic review and meta-analysis that included 18 studies (15 retrospective and three prospective) and 7680 images of colorectal polyps was performed to evaluate the prediction of histology[39]. AI had an accuracy of 0.96 (AUC), with a sensitivity of 92.3% and a specificity of 89.8%. Comparing studies that used NBI with those that used non-NBI techniques, they found a significantly better accuracy (AUC 0.98 vs 0.84) in the NBI studies. When they evaluated performance in the histological characterization of diminutive polyps, they found that the accuracy overlapped and that the pooled negative predictive value was 91.3%-95.1%, which complies with the threshold of the "resect and discard" strategy of the ASGE. The difference in the performance of AI and endoscopists was not significant except in studies that included nonexpert endoscopists, in which the performance AI was significantly better than that of nonexpert endoscopists.

Endocytoscopy (EC) is an advanced endoscopic technique in which a contact light microscopy system is integrated into the distal tip of a conventional colonoscope[40]. This technique has performed well in differentiating between diminutive neoplastic and non-neoplastic colorectal polyps both alone[41] and in combination with NBI (EC-NBI)[42]. The combined use of EC and CAD (EC-CAD) to classify the histology of colorectal polyps has been studied. According to Mori et al[40], this technology achieved a sensitivity of 92%, a specificity of 79.5%, and an accuracy of 89.2%. The performance was comparable to EC image evaluation by experts, but significantly better than evaluation by trainees for identifying neoplastic polyps.

AI in IBD

Ulcerative colitis (UC) is a chronic inflammatory disease, and the therapeutic goals have changed over the last decades, from the treatment of symptoms toward mucosal healing, with the aim of modifying the natural history of disease[43-45]. Mucosal healing can be defined by endoscopic evaluation because the inflammation is limited to the mucosal layer[44]. However, it is currently known that histological evidence of inflammation, even in patients with mucosal healing, is a prognostic marker of a two-to-three-fold increase in the risk of colitis relapse and colon cancer[44,45]. Consequently, assessment of inflammatory activity and severity are important for the choice of therapeutic and follow-up strategies. This supports performing colonoscopies with biopsies and visually interpretation of mucosal and histologic parameters of inflammation using standardized disease activity measures[46]. Visual interpretation is subject to bias, interobserver, and intraobserver variability. Histological evaluation requires time for processing and interpretation, which limits the ability to make decisions in real time[46].

Several studies have used AI for the macroscopic and histological characterization of colic mucosa in UC. Takenaka et al[47] developed a deep neural network for evaluation of UC with the aim of predicting endoscopic remission, a UC endoscopic index of severity score of 0; and histologic remission, a Geboes score of ≤ 3 points using endoscopic images. In the training set, they used 40,758 images of colonoscopies and the histologic results of 6885 biopsies from the same patients. They validated the results in a prospective study of 875 patients with 4187 endoscopic images and 4104 biopsy specimens. In the validation phase, the AI system was able to identify patients with endoscopic remission with 90.1% of accuracy and histologic remission with 92.9% of accuracy using only endoscopic images. The results suggest that computer-aided diagnosis can reliably assist endoscopic and histologic disease activity in UC without the need for mucosal biopsies. It should be noted that the study used only white-light images (WLIs). Chromoendoscopy was not used after WLI evaluation.

However, there are doubts that conventional WLI can reliably identify persistent histologic inflammation. Alternatives such as NBI, confocal laser endomicroscopy, or EC have emerged[48]. Despite their excellent diagnostic performance, these techniques are highly dependent on experienced endoscopists[48]. Maeda et al[48] developed a CAD system that used AI and EC to predict persistent histologic inflammation (Figure 3). It was used to retrospectively evaluate 12900 EC images for ML and 9935 EC images for validation, in addition assessment of biopsy samples by experienced pathologists blinded to the endoscopy results. The algorithm had 74% sensitivity, 97% specificity, and 91% accuracy in identifying histologic inflammation, and perfect reproducibility. The results are promising, and the authors concluded that the symbiosis achieved with the CAD system and EC has the potential to support immediate intervention and ultimately reduce the number of required biopsy samples.

Figure 3 Application of computer-aided detection systems to endocytoscopy. A, D: Conventional endoscopic images in WLI; B, E: Endocytoscopic images; C, F: Histological images. A CAD system evaluated the endocytoscopy image B: “healing” and E: active inflammation. The classifications were later confirmed by the pathologist[48]. Citation: Maeda Y, Kudo SE, Mori Y, Misawa M, Ogata N, Sasanuma S, Wakamura K, Oda M, Mori K, Ohtsuka K. Fully automated diagnostic system with artificial intelligence using endocytoscopy to identify the presence of histologic inflammation associated with ulcerative colitis (with video). Gastrointest Endosc 2019; 89(2): 408-415. Copyright © The Author(s) 2019 by the American Society for Gastrointestinal Endoscopy. Published by Elsevier, Inc.

There is a lack of published studies of the potential benefit of AI in assessment of disease activity in Crohn’s disease. Evidence of the applicability of AI to the surveillance of dysplasia in patients with IBD is still lacking. Maeda et al[49] described a case in which an AI system assisted in the detection of dysplastic lesions in a patient with UC. They concluded that AI may have potential in this context, especially for nonexpert endoscopists.

CONCLUSION

The evidence for a role of AI in diagnostic endoscopy is growing, with increasing supporting evidence in dysplasia detection in BE, identification and assessment of colorectal polyps, and characterization of colonic mucosa in UC. A number of possible applications are still under evaluation (Table 1)[50-58]. Deus ex machina is a Latin expression meaning “God from the machine”. The term was coined from the conventions of ancient Greek theatre, where actors who were playing gods were brought onto stage using a machine[59]. Later it was used in the story plots of theatre plays and movies to describe un unexpected and sudden solution to resolve main issues or tragedies. AI is currently being developed for use in endoscopy daily practice. It seems to be a quick solution of the long-standing problems of the interobserver agreement and the subjective “feeling” and accuracy of the endoscopist in diagnostic exams. Apart from the complexity of improving or overcoming human evaluation, there are also availability issues that will limit the access to this new technology.

Table 1 Potential applications of artificial intelligence in gastrointestinal pathology (under evaluation).

Disease	Detection	Ref.
Esophageal squamous cell carcinoma (ESCC)	Detection of early ESCC	Yang et al[50]
		Kumagai et al[51]
Helicobacter pylori infection	Diagnosis of Helicobacter pylori infection	Mohan et al[52]
Gastric cancer (GC)	Detection of preneoplastic lesions and early gastric cancerPrediction the invasion depth of GC	Anta et al[53]
		Nagao et al[54]
Celiac disease	Early detection of celiac disease	Sana et al[55]
Gastrointestinal bleeding	Detection of small bowel bleeding with wireless capsule endoscopic	Aoki et al[56]
		Tsuboi et al[57]
Pancreatic lesions	Differentiation of pancreatic cancer from chronic pancreatitis and normal pancreas	Tonozuka et al[58]

In the meantime, the added value of electronic systems to human practice must not be underrated as they would definitely contribute to the standardization of diagnostic endoscopy worldwide. Apart from that, savings in the time, effort, economic, and human resources needed for diagnostic exams, would make room for improvements in endoscopic therapeutic procedures, which are growing in complexity over time.

In conclusion, AI is currently used to optimize diagnostic endoscopy procedures, with growing indications and evidence over time. Although it may not be ready for prime-time use worldwide, the future of AI is near. It will depend on us, human endoscopists, to define how far these machines can help us with (and solve?) our naturally imperfect issues.