Optical diagnosis of colorectal polyps using convolutional neural networks

Figure 1 The relationship between convolutional neural networks, deep learning, machine learning and artificial intelligence.

Figure 2 Illustration of coloured heatmaps, overlaid to the polyp, which demonstrates the regions that most likely contributed to the convolutional neural networks’s diagnosis. A, B, C, D: Original narrow band imaging (NBI) of polyps; a, b, c, d: Coloured heatmap overlaid on the NBI image; Red: Higher probability that this region informed the convolutional neural networks (CNN)’s diagnosis; Blue: Lower probability that this region informed the CNN’s diagnosis. Images adapted and modified with permission from the publisher[31]. Citation: Jin EH, Lee D, Bae JH, Kang HY, Kwak MS, Seo JY, Yang JI, Yang SY, Lim SH, Yim JY, Lim JH, Chung GE, Chung SJ, Choi JM, Han YM, Kang SJ, Lee J, Chan Kim H, Kim JS. Improved Accuracy in Optical Diagnosis of Colorectal Polyps Using Convolutional Neural Networks. Gastroenterology 2020; 158(8): 2169-2179. Copyright© The Authors 2020. Published by Elsevier.

Figure 3 Spatial colour coded histology map which allows the user to visualise the sub-regions of the polyp surface that contributed to the convolutional neural networks’s decision process. A: Hyperplastic polyps; B: Adenomatous polyps; C: Sessile serrated lesions; Red: High-confidence neoplastic diagnosis; Green: High-confidence non-neoplastic diagnosis; Yellow: Indeterminate or low-confidence diagnosis. Adapted from Ref. [28]. Citation: Rodriguez-Diaz E, Baffy G, Lo WK, Mashimo H, Vidyarthi G, Mohapatra SS, Singh SK. Real-time artificial intelligence-based histologic classification of colorectal polyps with augmented visualization. Gastrointest Endosc 2021; 93: 662-670. Copyright© The Authors 2021. Published by Elsevier.