Published online Mar 8, 2024. doi: 10.37126/aige.v5.i1.90574
Peer-review started: December 7, 2023
First decision: December 29, 2023
Revised: January 11, 2024
Accepted: February 2, 2024
Article in press: February 2, 2024
Published online: March 8, 2024
Artificial intelligence (AI) has potential in the optical diagnosis of colorectal polyps.
To evaluate the feasibility of the real-time use of the computer-aided diagnosis system (CADx) AI for ColoRectal Polyps (AI4CRP) for the optical diagnosis of diminutive colorectal polyps and to compare the performance with CAD EYETM (Fujifilm, Tokyo, Japan). CADx influence on the optical diagnosis of an expert endoscopist was also investigated.
AI4CRP was developed in-house and CAD EYE was proprietary software provided by Fujifilm. Both CADx-systems exploit convolutional neural networks. Colorectal polyps were characterized as benign or premalignant and histopathology was used as gold standard. AI4CRP provided an objective assessment of its characterization by presenting a calibrated confidence characterization value (range 0.0-1.0). A predefined cut-off value of 0.6 was set with values < 0.6 indicating benign and values ≥ 0.6 indicating premalignant colorectal polyps. Low confidence characterizations were defined as values 40% around the cut-off value of 0.6 (< 0.36 and > 0.76). Self-critical AI4CRP’s diagnostic performances excluded low confidence characterizations.
AI4CRP use was feasible and performed on 30 patients with 51 colorectal polyps. Self-critical AI4CRP, excluding 14 low confidence characterizations [27.5% (14/51)], had a diagnostic accuracy of 89.2%, sensitivity of 89.7%, and specificity of 87.5%, which was higher compared to AI4CRP. CAD EYE had a 83.7% diagnostic accuracy, 74.2% sensitivity, and 100.0% specificity. Diagnostic performances of the endoscopist alone (before AI) increased non-significantly after reviewing the CADx characterizations of both AI4CRP and CAD EYE (AI-assisted endoscopist). Diagnostic performances of the AI-assisted endoscopist were higher compared to both CADx-systems, except for specificity for which CAD EYE performed best.
Real-time use of AI4CRP was feasible. Objective confidence values provided by a CADx is novel and self-critical AI4CRP showed higher diagnostic performances compared to AI4CRP.
Core Tip: In this study, two computer-aided diagnosis systems (CADx) [Artificial intelligence for ColoRectal polyps (AI4CRP) and CAD EYE] were compared head-to-head and showed that real-time use was feasible in clinical practice, but does not yet meet quality standards for optical diagnosis. AI4CRP provided characterizations accompanied by confidence values, enabling self-critical AI4CRP in which low confidence characterizations were excluded. Self-critical AI4CRP resulted in considerably higher diagnostic performances compared to AI4CRP. The AI-assisted endoscopists, optically diagnosing colorectal polyps after reviewing both CADx characterizations, had non-significantly higher diagnostic performances compared to the endoscopist alone (before CADx).
- Citation: van der Zander QEW, Schreuder RM, Thijssen A, Kusters CHJ, Dehghani N, Scheeve T, Winkens B, van der Ende - van Loon MCM, de With PHN, van der Sommen F, Masclee AAM, Schoon EJ. Artificial intelligence for characterization of diminutive colorectal polyps: A feasibility study comparing two computer-aided diagnosis systems. Artif Intell Gastrointest Endosc 2024; 5(1): 90574
- URL: https://www.wjgnet.com/2689-7164/full/v5/i1/90574.htm
- DOI: https://dx.doi.org/10.37126/aige.v5.i1.90574
Endoscopists’ task in performing colonoscopies increasingly involves optical diagnosis, the endoscopic characterization of colorectal polyps. Recently, diagnostic performance of optical diagnosis increased due to optimization of technologies such as high definition imaging, magnification, and image enhancement techniques like blue light imaging (BLI)[1,2]. Despite these optimizations, endoscopists do not consistently meet quality standards set by the American society for gastrointestinal endoscopy (ASGE) and the European society of gastrointestinal endoscopy (ESGE) for implementation of the resect-and-discard and diagnose-and-leave strategies based on optical diagnosis[3,4]. The first strategy entails diminutive (≤ 5 mm) colorectal polyps to be resected and discarded without histopathological assessment under the condition of a ≥ 90% agreement in the post-polypectomy surveillance interval between the optical and histopathological diagnosis. The second strategy states that diminutive hyperplastic polyps in the rectosigmoid can be left in situ if a negative predictive value (NPV) of ≥ 90% is reached for the optical diagnosis of adenomatous polyps. Large, multicenter studies demonstrated disappointing results on optical diagnosis, even for additionally trained (bowel cancer screening) endoscopists, hampering implementation in clinical practice[5,6]. Diagnostic performances are operator dependent, showing high interobserver variability, and rely on training and expertise[3,7,8].
Optical diagnosis with artificial intelligence (AI) has the potential to overcome this high interobserver variability by minimizing the operator dependence and providing objective optical diagnoses[9]. Accurate characterization of colorectal polyps with computer-aided diagnosis systems (CADx) may facilitate the implementation of the resect-and-discard and diagnose-and-leave strategies by meeting the set quality standards. Implementation of these strategies may lead to a reduction in unnecessary polypectomies, thereby decreasing the risk of post-polypectomy complications, reducing histopathology costs, and improving the cost-effectiveness of colonoscopy[10,11].
The primary aim of this study was to evaluate the feasibility of the real-time use of the CADx AI for ColoRectal polyps (AI4CRP) for the optical diagnosis of diminutive (≤ 5 mm) colorectal polyps. Secondary aims were a head-to-head comparison of AI4CRP with CAD EYETM (Fujifilm, Tokyo, Japan), evaluating the diagnostic performances of self-critical AI4CRP (providing only high confidence diagnoses), the diagnostic performances of an expert endoscopist (endoscopist alone), and the influence of CADx on the optical diagnosis of an expert endoscopist (AI-assisted endoscopist).
This prospective study was conducted at the Catharina Hospital Eindhoven, the Netherlands. The study was performed in accordance with the declaration of Helsinki and the General Data Protection Regulation. The Medical Research Ethics Committees United (W20.239, July 2021) approved the study (ClinicalTrials.gov NCT05349110).
AI4CRP, developed in-house by our research group (Video Coding & Architectures, Eindhoven University of Technology, the Netherlands), is an image-based CADx exploiting convolutional neural networks. AI4CRP was previously validated and a technical explanation has been published[12,13]. AI4CRP was trained multicenter and tested using prospectively collected datasets from multiple endoscopy vendors (Fujifilm and Pentax) (Supplementary Table 1). Characterizations were made in three image modalities: high-definition white light (HDWL), BLI, and linked color imaging (LCI). AI4CRP characterized colorectal polyps as benign (hyperplastic polyp) or premalignant [adenoma and sessile serrated lesion (SSL)]. The characterization was provided by a green assist bar for benign and a red assist bar for premalignant polyps. A heatmap pointed out the area of interest (Figure 1). Distinctive from other CADx-systems, AI4CRP provided calibrated confidence characterization values (range 0.0-1.0) representing the objective confidence level of AI4CRP in its characterization. A predefined cut-off value of 0.6 was set with values < 0.6 indicating benign and values ≥ 0.6 indicating premalignant colorectal polyps. A value close(r) to 0.0 implies high confidence for a benign colorectal polyp and a value close(r) to 1.0 high confidence for a premalignant colorectal polyp. Providing these confidence values enabled a self-critical AI4CRP in which low confidence characterizations were excluded. Low confidence characterizations were defined as values 40% around the cut-off value of 0.6 (< 0.36 and > 0.76). To explore the added value of our self-critical CADx and allowing for an exploration of self-critical AI4CRP, the system was compared head-to-head with CAD EYE.
CAD EYE is a commercial, video-based CAD-system developed to detect and characterize colorectal polyps. CAD EYE exploits convolutional neural networks[11]. For this study, only the characterization mode (BLI) was used. CAD EYE characterized colorectal polyps as hyperplastic (hyperplastic polyp and SSL) or neoplastic (adenoma) (note the difference in SSL characterization compared to AI4CRP). A status bar indicated the status of the characterization (complete or incomplete), a visual assist circle colored green for hyperplastic and yellow for neoplastic, a position map indicated the position of the colorectal polyp, and a characterization was displayed (Figure 1)[14].
Patients, aged ≥ 18 years, referred for screening colonoscopies, symptoms, or surveillance were eligible for participation. Consecutive patients were included if at least one diminutive colorectal polyp was encountered. Exclusion criteria were polyposis syndromes, inflammatory bowel diseases, inadequate bowel preparations (Boston bowel preparation scale < 6), and emergency colonoscopies. Patients were informed during a screening visit at the outpatient clinic before the colonoscopy. All patients provided written informed consent.
Colonoscopies were performed by one expert endoscopist (R.M.S.). The endoscopist was additionally trained in optical diagnosis (succeeding several training sessions in optical diagnosis organized by the ESGE), performed optical diagnoses on a regular basis according to the ESGE curriculum for optical diagnosis[1], and is a teacher in optical diagnosis training sessions. The endoscopist was familiarized with both CADx-systems. He was involved in the development of AI4CRP and used CAD EYE in clinical practice for 6 months before the start of this study. A maximum of three diminutive colorectal polyps per patient were included due to time restrictions. If more than three diminutive colorectal polyps were encountered, the first three were included to minimize selection bias. The endoscopist optically diagnosed colorectal polyps real-time (endoscopists alone) as benign (hyperplastic polyp) or premalignant (adenoma and SSLs) using BLI and according to Japan NBI Expert Team and BLI adenoma serrated international classification (BASIC)[15,16]. The endoscopist provided a confidence level [low or high (≥ 90%)] for each optical diagnosis. Subsequently, all colorectal polyps optically diagnosed by the endoscopist were characterized by both CADx-systems in sequence. AI4CRP characterized images captured from the real-time video output in each image modality separately and calculated an overall characterization using all three image modalities (multimodal imaging). Images were manually captured by a research physician. Motion-blurred images and images out-of-focus were excluded. Afterwards, CAD EYE was activated by the endoscopist to provide a characterization. Both CADx characterizations were recorded and saved. Lastly, the endoscopist optically diagnosed the colorectal polyps after reviewing both CADx characterizations (AI-assisted endoscopist) and again provided a confidence level.
Despite proper endoscope positioning, CAD EYE provided inconclusive characterizations, defined as unstable characterizations over time (switching diagnoses between hyperplastic and neoplastic) despite a complete status bar. The video-recorded CAD EYE characterizations were assessed by two independent expert endoscopists blinded to histopathology. Upon agreement of inconclusiveness, these characterizations were excluded from the analyses.
The primary outcome was the feasibility of the real-time use of AI4CRP. Feasibility was defined as seamless video output reception from the endoscopy processor without noticeable clinically relevant latency (the time from capturing the endoscopic image to outputting the analyzed results)[17] and seamless operation of the software in obtaining characterizations. Latency was not measured by AI4CRP itself or by the investigators since it is known from previous studies that small differences in latency were not noticeable for endoscopists, and therefore only clinically noticeable latency was deemed relevant[12]. Secondary outcomes were real-time diagnostic performances of (self-critical) AI4CRP and a head-to-head comparison of (self-critical) AI4CRP with CAD EYE and an expert endoscopist (endoscopist alone and AI-assisted endoscopist). Histopathology was used as gold standard and assessed according to the revised Vienna classification. Involved pathologists were specialized in gastrointestinal histopathology. Differences in characterization of SSLs by AI4CRP, CAD EYE, and the endoscopist were accounted for by histopathology in computing measures of diagnostic performance. Outcomes were reported according to the STARD (standard for reporting diagnostic accuracy studies) checklist.
Due to the feasibility design of the study, no formal sample size calculation was performed. The sample size (n = 30 patients) was based on a previous CADx feasibility study[18]. Baseline characteristics are presented as proportions (%) for categorical variables or as mean [standard deviation (SD)] for numerical variables. Feasibility was described qualitatively. Diagnostic performances were investigated in terms of diagnostic accuracy, sensitivity, specificity, and negative and positive predictive values (NPV, PPV), expressed with 95% confidence intervals. As sensitivity analysis, cluster bootstrapping was performed to account for multiple colorectal polyps per patient. Self-critical AI4CRP was analyzed post-hoc. Differences between (self-critical) AI4CRP, CAD EYE, and the endoscopist were analyzed using the McNemar test for paired proportions. Two-sided P values ≤ 0.05 were considered statistically significant. Statistical analyses were performed with IBM SPSS Statistics (IBM Corp., United States) and R (R Foundation, Austria). The statistical methods of this study were reviewed by B. Winkens from the Department of Methodology and Statistics of Maastricht University.
Patients who underwent a colonoscopy at Catharina Hospital Eindhoven between August and November 2021 were screened for eligibility. In total, 30 patients with 51 colorectal polyps were included (Figure 2). Patient characteristics are provided in Table 1. Mean polyp size was 2.8 mm (SD 1.0). Histopathology showed 32 tubular adenomas (62.7%), 1 tubulovillous adenoma (2.0%), 6 SSLs (11.8%), and 12 hyperplastic polyps (23.5%) (Table 2).
| Patients, n = 30 | ||
| Gender, female | 13 (43.3) | |
| Age in years, mean (SD) [range] | 65.8 (8.4) [50-78] | |
| Indication colonoscopy | ||
| Bowel cancer screening program | 15 (50.0) | |
| Surveillance | 10 (33.3) | |
| Symptoms | 5 (16.7) | |
| Family history positive for CRC | 5 (16.7) | |
| BBPS, mean (SD) | 6.6 (1.4) | |
| Number of colorectal polyps per patient1 | ||
| 1 colorectal polyp | 15 (50.0) | |
| 2 colorectal polyps | 9 (30.0) | |
| 3 colorectal polyps | 6 (20.0) | |
| Colorectal polyps, n = 51 | ||
| Location | ||
| Cecum | 7 (13.7) | |
| Ascending colon | 8 (15.7) | |
| Transverse colon | 15 (29.4) | |
| Descending colon | 5 (9.8) | |
| Sigmoid | 10 (19.6) | |
| Rectum | 6 (11.8) | |
| Size, mean (SD) [range] | 2.8 (1.0) [2-5] | |
| Morphology | ||
| Sessile (Paris Is) | 45 (88.2) | |
| Flat-elevated (Paris IIa) | 6 (11.8) | |
| Histopathology | ||
| Tubular adenoma, LGD | 32 (62.7) | |
| Tubulovillous adenoma, LGD | 1 (2.0) | |
| Sessile serrated lesion, no dysplasia | 6 (11.8) | |
| Hyperplastic polyp, no dysplasia | 12 (23.5) | |
| Resection technique – cold snare | 51 (100.0) | |
Real-time use of AI4CRP was deemed feasible in clinical practice. By means of plug-and-play AI4CRP was easily connected to the endoscopy processor. No noticeable clinically relevant latency was observed in receiving the video output from the processor and the software worked flawless without interruptions.
AI4CRP was able to characterize all 51 colorectal polyps. Eight images were excluded because the images were out of focus and four because of motion blur. For these colorectal polyps, a second image was taken by the endoscopist. AI4CRP showed a sensitivity of 82.1% (95%CI 0.66-0.92) and a diagnostic accuracy of 80.4% (95%CI 0.66-0.90) in BLI, which was significantly higher compared to HDWL (sensitivity 59.0%, P = 0.022 and diagnostic accuracy 66.7%, P = 0.007) (Table 3, Supplementary Figure 1). NPV was also highest in BLI (56.3%, 95%CI 0.31-0.79), but not significantly different from other image modalities. Self-critical AI4CRP excluded 14 low confidence characterizations [27.5% (14/51), tubular adenomas n = 7, SSLs n = 3, hyperplastic polyps n = 4]. Self-critical AI4CRP showed higher diagnostic performances on all metrics compared to AI4CRP (sensitivity 89.7% and diagnostic accuracy 89.2%) (Table 4).
| AI4CRP (n = 51) | ||||
| BLI, % (95%CI) | HDWL, % (95%CI) | LCI, % (95%CI) | Multimodal imaging, % (95%CI) | |
| Sensitivity | 82.1 (0.66-0.92) | 59.0 (0.42-0.74) | 76.9 (0.60-0.88) | 71.8 (0.55-0.84) |
| Specificity | 75.0 (0.43-0.93) | 91.7 (0.60-1.00) | 83.3 (0.51-0.97) | 91.7 (0.60-1.00) |
| PPV | 91.4 (0.76-0.98) | 95.8 (0.77-1.00) | 93.8 (0.78-0.99) | 96.6 (0.80-1.00) |
| NPV | 56.3 (0.31-0.79) | 40.7 (0.23-0.61) | 52.6 (0.29-0.75) | 50.0 (0.29-0.71) |
| Diagnostic accuracy | 80.4 (0.66-0.90) | 66.7 (0.52-0.79) | 78.4 (0.64-0.88) | 76.5 (0.62-0.87) |
| AI4CRP1, % (95%CI), n = 51 | Self-critical AI4CRP1, % (95%CI), n = 37 | CAD EYE, % (95%CI), n = 49 | Endoscopist alone2, % (95%CI), n = 47 | AI-assisted endoscopist2,3, % (95%CI), n = 49 | |
| Sensitivity | 82.1 (0.66-0.92) | 89.7 (0.72-0.97) | 74.2 (0.55-0.87) | 97.4 (0.85-1.00) | 97.4 (0.85-1.00) |
| Specificity | 75.0 (0.43-0.93) | 87.5 (0.47-0.99) | 100.0 (0.78-1.00) | 77.8 (0.40-0.96) | 90.9 (0.57-1.00) |
| PPV | 91.4 (0.76-0.98) | 96.3 (0.79-1.00) | 100.0 (0.82-1.00) | 94.9 (0.81-0.99) | 97.4 (0.85-1.00) |
| NPV | 56.3 (0.31-0.79) | 70.0 (0.35-0.92) | 69.2 (0.48-0.85) | 87.5 (0.47-0.99) | 90.9 (0.57-1.00) |
| Diagnostic accuracy | 80.4 (0.66-0.90) | 89.2 (0.74-0.96) | 83.7 (0.70-0.92) | 93.6 (0.81-0.98) | 95.9 (0.85-0.99) |
CAD EYE was able to provide a characterization for all but two colorectal polyps (n = 49, 96.1%), which were diagnosed inconclusively. CAD EYE had a sensitivity of 74.2% (95%CI 0.55-0.87), a specificity of 100.0% (95%CI 0.78-1.00), a NPV of 69.2% (95%CI 0.48-0.85), and a diagnostic accuracy of 83.7% (95%CI 0.70-0.92) (Table 4).
The endoscopist (endoscopist alone) optically diagnosed 47 (92.2%, 47/51) colorectal polyps with high confidence. Before AI (endoscopist alone), sensitivity was 97.4% (95%CI 0.85-1.00), specificity 77.8% (95%CI 0.40-0.96), NPV 87.5% (95%CI 0.47-0.99), and diagnostic accuracy 93.6% (95%CI 0.81-0.98) (Table 4). Although this study was not powered to detect a difference between the endoscopist alone and the AI-assisted endoscopist, after reviewing characterizations of both CADx-systems specificity, PPV, NPV, and diagnostic accuracy increased non-significantly for the AI-assisted endoscopist (Table 4, Supplementary Figure 2). The number of optical diagnoses made with high confidence also increased [endoscopist alone 92.2% (47/51) vs AI-assisted endoscopists 96.1% (49/51), P = 0.500] (Supplementary Table 2).
Diagnostic performances of the AI-assisted endoscopist were higher compared to both CADx-systems, except for specificity for which CAD EYE performed best. Comparing diagnostic performances of AI4CRP with the endoscopist alone showed a significantly higher sensitivity (P = 0.031) and a non-significantly higher specificity and diagnostic accuracy for the endoscopist (P = 1.000 and P = 0.180, respectively) (Supplementary Figure 2). The AI-assisted endoscopist also had a significantly higher sensitivity (P = 0.031) and a non-significantly higher specificity and diagnostic accuracy (P = 0.500 and P = 0.289, respectively) than AI4CRP. Self-critical AI4CRP did not show any significant difference with the endoscopist alone and the AI-assisted endoscopist. Compared with CAD EYE, the endoscopist alone had a significantly higher diagnostic accuracy (P = 0.004) and sensitivity (P = 0.016), while specificity was non-significantly lower (P = 0.500) (Supplementary Figure 2). The same accounted for the comparison between CAD EYE and the AI-assisted endoscopist. Performing cluster bootstrapping to correct for multiple colorectal polyps per patient did not change the conclusions (Supplementary Table 3). Analysis according to colorectal polyp location are presented in Supplementary Table 4.
AI4CRP use for the optical diagnosis of diminutive colorectal polyps was feasible and showed promising results. The novelty of our AI4CRP lies in providing objective confidence values. Self-critical AI4CRP achieved considerably higher diagnostic performances compared to AI4CRP. Reviewing characterizations by AI4CRP and CAD EYE did non-significantly increase the performance of the AI-assisted endoscopist.
Real-time use of AI4CRP was feasible and did not obstruct clinical workflow. No clinically relevant time delays in obtaining CADx characterizations were observed. This study compared two CADx-systems head-to-head, namely AI4CRP and CAD EYE. By comparing a commercially available CADx with an in-house developed CADx, comparison between the systems and a self-critical system was possible. Diagnostic performances of both CADx-systems were non-significantly inferior compared to the performance of the expert endoscopist, with the exception of specificity, were CAD EYE demonstrated the best performance. This difference in specificity between (self-critical) AI4CRP and CAD EYE, can be explained by the differences in characterizing SSLs. Performances of both CADx-systems should be improved for utility in clinical practice.
Objective assessment of the confidence level as performed by self-critical AI4CRP is a novelty. Diagnostic performances were considerably higher for self-critical AI4CRP compared to AI4CRP. CAD EYE does not provide a confidence value while inconclusive diagnoses occurred (3.9%). However, these inconclusive diagnoses were marked by expert consensus and are not objective as for self-critical AI4CRP. Rondonotti et al[19] reported higher numbers of CAD EYE characterizations being unstable over time (7.9%) or not possible (1.3%). Self-critical AI4CRP made low confidence characterizations in 27.5%. Providing an objective confidence level can be seen as a form of explainable AI which may increase endoscopists' trust in CADx and therefore has potential applicability in real-time endoscopy practice. At the same time, one can argue that CADx should be of added value particularly in colorectal polyps deemed difficult by endoscopists. Interestingly, the low confidence diagnoses made by the endoscopist were high confidence diagnoses by AI4CRP in 75.0% of cases. Furthermore, self-critical AI4CRP was performed post-hoc. In real-time colonoscopy, endoscopists could do another attempt in gaining a high confidence characterization by repositioning the colonoscope and thereby optimize the endoscopic imaging possibly lowering the number of low confidence characterizations. Future studies should investigate if defining low confidence characterizations as diagnosis with a confidence value of 40% around the cut-off value is sufficient.
CADx utility in clinical practice will not be in a stand-alone fashion, but in aiding endoscopists. A strength of this study is the AI-assisted performances of the endoscopist, in contrast to previous studies in which endoscopist alone or AI-assisted non-expert endoscopist vs CADx were investigated[11,20]. The non-significant increase between the diagnostic performances of the endoscopist alone and the AI-assisted endoscopist is comparable with results of Hassan et al[21]. Furthermore, Jin et al[22] only showed an increase for non-experts and not for experts. Here, the number of optical diagnoses made with high confidence did increase for AI-assisted optical diagnosis.
Most CADx-systems have been trained to operate in a single image enhancement modality, i.e. narrow band imaging (NBI) or BLI. Zachariah et al[23] and Biffi et al[24] trained their systems using both HDWL and NBI or BLI, respectively. They compared the diagnostic performances of their CADx in HDWL with the performances in the image enhancement modality and found no significant differences. This favors the use of HDWL since the interpretation of image enhancement modalities requires training[25], limits generalizability, and hampers the utility of AI-assisted CADx by undertrained endoscopists. A strength of our study is that AI4CRP was trained with multiple image enhancement modalities, namely HDWL, BLI, LCI, and i-scan 1, 2, and 3. In contrast to Zachariah et al[23] and Biffi et al[24], AI4CRP’s diagnostic performances were significantly higher in BLI compared to HDWL. Future research should, therefore, investigate the effect of different image enhancement modalities (especially BLI) on the output of CADx compared to HDWL.
Self-critical AI4CRP and CAD EYE reached a NPV of ≥ 90% for rectosigmoid polyps according to the quality standard for the diagnose-and-leave strategy by the ASGE[3]. Both CADx-systems also met the quality standard of the ESGE for the diagnose-and-leave strategy and self-critical AI4CRP also the ESGE quality standard for the resect-and-discard strategy[4]. Previously, CAD EYE and GI-Genius (Medtronic, United States) also met the PIVI quality standards[26,27]. Although the risk of cancer in diminutive colorectal polyps is very low, misdiagnosis does pose risks when leaving adenomatous polyps in situ[28,29]. In an international survey, two-thirds of endoscopists considered implementation of resect-and-discard not feasible because of the fear of making incorrect optical diagnoses[30]. Studies should investigate if this fear of incorrect optical diagnosis may be leveraged with CADx.
No consensus exists on the characterization of SSLs between different CADx-systems. Where CAD EYE, a CADx by Sánchez-Montes et al[31], and Zachariah et al[24] characterizes SSLs as hyperplastic, other systems excluded SSLs[9,11,32-34]. Rondonotti et al[19] marginalized the clinical relevance of SSLs because of their low prevalence among diminutive rectosigmoid polyps. Albeit this low prevalence, given that SSLs bear a malignant potential[35,36], differentiating them from hyperplastic polyps is advocated and promotes clinical utility of CADx. AI4CRP, characterizing SSLs as premalignant, pursued to do just that because of the high need of improving SSL diagnosis[31]. Expanding CADx characterizations to multiple-class characterizations, allowing for the separate diagnosis of SSLs, more in line with clinical practice, would facilitate CADx implementation into clinical practice even further.
The main strength of this study is the head-to-head comparison of two CADx-systems characterizing the same colorectal polyps in sequence. Certain limitations of our study should also be acknowledged. Due to the feasibility design, no formal sample size calculation was performed and the number of included colorectal polyps was limited. Both CADx-systems were compared with only one expert endoscopist and testing was performed single center, limiting generalizability. AI4CRP is an image-based CADx, whereas CAD EYE is video-based. Both systems characterized the same colorectal polyps in a sequential approach rather than a parallel approach. The sequential approach led to both CADx-systems analyzing slightly different colorectal polyp frames potentially introducing bias. Bias could also have occurred since AI4CRP was trained with data from the same hospital in which it was tested in this study, possibly favoring AI4CRP performances, while this is not true for CAD EYE. An important limitation was the semi-automated use of AI4CRP. Images had to be manually captured by a research physician, limiting functional use of AI4CRP in clinical practice. A fully automated approach is currently under development. Furthermore, images out of focus or motion blurred imaged were excluded and a new image had to be taken. Although inconvenient, this only hampered the work flow minimally, but could have introduced bias. An image quality indicator alongside the CADx characterization, could be helpful in quantifying and reducing this bias.
In conclusion, real-time use of AI4CRP was feasible and achieved promising results. Self-critical AI4CRP, excluding low confidence characterizations, showed increased diagnostic performances compared to AI4CRP. The objective assessment of the confidence level is a novelty with great potential applicability in real-time endoscopy practice. Diagnostic performances of the AI-assisted endoscopist, after reviewing both CADx characterizations, were higher compared to both CADx-systems. Diagnostic performances of the AI-assisted endoscopist were non-significantly superior to the endoscopists alone. In the future, larger sized studies should expand on our findings.
The importance of optical diagnosis, the endoscopic characterization of colorectal polyps, increases. However, correct endoscopic characterization and differentiation between benign and premalignant polyps remains difficult even for experienced endoscopists.
The ability of modern-day computer-aided diagnosis systems (CADx) to automatically recognize informative patterns in datasets can potentially improve accurate characterization of colorectal polyps and facilitate the implementation of treatment strategies based on optical diagnosis by meeting set quality standards.
Aim of this study was to evaluate the feasibility of the real-time use of the in-house developed CADx-system artificial intelligence for ColoRectal polyps (AI4CRP) for the optical diagnosis of diminutive (≤ 5 mm) colorectal polyps. Secondary aims were a head-to-head comparison of AI4CRP with CAD EYETM (Fujifilm, Tokyo, Japan), evaluating the diagnostic performances of self-critical AI4CRP (providing only high confidence diagnoses), the diagnostic performances of an expert endoscopist (endoscopist alone), and the influence of CADx on the optical diagnosis of an expert endoscopist [artificial intelligence (AI)-assisted endoscopist].
The two CADx-systems (AI4CRP and CAD EYE) were compared head-to-head. Colorectal polyps were characterized as benign or premalignant and histopathology was used as gold standard. AI4CRP provided characterizations accompanied by confidence values, enabling self-critical AI4CRP in which low confidence characterizations were excluded. The AI-assisted endoscopists, optically diagnosed colorectal polyps after reviewing both CADx characterizations.
Real-time use of AI4CRP was deemed feasible in clinical practice. AI4CRP showed a sensitivity of 82.1%, a specificity of 75.0%, a negative predictive value of 56.3%, and a diagnostic accuracy of 80.4%. Self-critical AI4CRP excluded 14 low confidence characterizations, resulted in considerably higher diagnostic performances compared to AI4CRP. CAD EYE had a sensitivity of 74.2%, a specificity of 100.0%, a negative predictive value of 69.2%, and a diagnostic accuracy of 83.7%. Diagnostic performances of the endoscopist alone (before AI) increased non-significantly after reviewing the CADx characterizations of both AI4CRP and CAD EYE (AI-assisted endoscopist). Diagnostic performances of the AI-assisted endoscopist were higher compared to both CADx-systems, except for specificity for which CAD EYE performed best.
Real-time use of AI4CRP was feasible. Objective confidence values provided by a CADx is novel and self-critical AI4CRP showed higher diagnostic performances compared to AI4CRP. Reviewing characterizations by AI4CRP and CAD EYE did not increase the performance of the AI-assisted endoscopist.
Future studies should expand on our findings and further investigate the added value of self-critical CADx-systems.
We thank the COMET-OPTICAL consortium for participation in this project: R.J.J. de Ridder, C.V. Hoge, J.M. Conchillo, J.J.L. Haans, R.W.M. Schrauwen, E.T.P. Keulen, E.J.A. Rondagh, and J.W.A. Straathof.
Provenance and peer review: Unsolicited article; Externally peer reviewed.
Peer-review model: Single blind
Corresponding Author's Membership in Professional Societies: Dutch Society of Gastroenterology, 17913; European Society of Gastrointestinal Endoscopy, 69078401.
Specialty type: Gastroenterology and hepatology
Country/Territory of origin: Netherlands
Peer-review report’s scientific quality classification
Grade A (Excellent): 0
Grade B (Very good): B
Grade C (Good): 0
Grade D (Fair): 0
Grade E (Poor): 0
P-Reviewer: Kader R, United Kingdom S-Editor: Liu JH L-Editor: A P-Editor: Cai YX
| 1. | Dekker E, Houwen BBSL, Puig I, Bustamante-Balén M, Coron E, Dobru DE, Kuvaev R, Neumann H, Johnson G, Pimentel-Nunes P, Sanders DS, Dinis-Ribeiro M, Arvanitakis M, Ponchon T, East JE, Bisschops R. Curriculum for optical diagnosis training in Europe: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy. 2020;52:899-923. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 39] [Cited by in F6Publishing: 44] [Article Influence: 11.0] [Reference Citation Analysis (0)] |
| 2. | Rondonotti E, Paggi S, Amato A, Mogavero G, Andrealli A, Conforti FS, Conte D, Spinzi G, Radaelli F. Blue-light imaging compared with high-definition white light for real-time histology prediction of colorectal polyps less than 1 centimeter: a prospective randomized study. Gastrointest Endosc. 2019;89:554-564.e1. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 22] [Cited by in F6Publishing: 18] [Article Influence: 3.6] [Reference Citation Analysis (0)] |
| 3. | ASGE Technology Committee; Abu Dayyeh BK, Thosani N, Konda V, Wallace MB, Rex DK, Chauhan SS, Hwang JH, Komanduri S, Manfredi M, Maple JT, Murad FM, Siddiqui UD, Banerjee S. ASGE Technology Committee systematic review and meta-analysis assessing the ASGE PIVI thresholds for adopting real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest Endosc. 2015;81:502.e1-502.e16. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 210] [Cited by in F6Publishing: 220] [Article Influence: 24.4] [Reference Citation Analysis (0)] |
| 4. | Houwen BBSL, Hassan C, Coupé VMH, Greuter MJE, Hazewinkel Y, Vleugels JLA, Antonelli G, Bustamante-Balén M, Coron E, Cortas GA, Dinis-Ribeiro M, Dobru DE, East JE, Iacucci M, Jover R, Kuvaev R, Neumann H, Pellisé M, Puig I, Rutter MD, Saunders B, Tate DJ, Mori Y, Longcroft-Wheaton G, Bisschops R, Dekker E. Definition of competence standards for optical diagnosis of diminutive colorectal polyps: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy. 2022;54:88-99. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 10] [Cited by in F6Publishing: 26] [Article Influence: 13.0] [Reference Citation Analysis (0)] |
| 5. | Rees CJ, Rajasekhar PT, Wilson A, Close H, Rutter MD, Saunders BP, East JE, Maier R, Moorghen M, Muhammad U, Hancock H, Jayaprakash A, MacDonald C, Ramadas A, Dhar A, Mason JM. Narrow band imaging optical diagnosis of small colorectal polyps in routine clinical practice: the Detect Inspect Characterise Resect and Discard 2 (DISCARD 2) study. Gut. 2017;66:887-895. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 119] [Cited by in F6Publishing: 126] [Article Influence: 18.0] [Reference Citation Analysis (0)] |
| 6. | Ahmad A, Moorghen M, Wilson A, Stasinos I, Haycock A, Humphries A, Monahan K, Suzuki N, Thomas-Gibson S, Vance M, Thiruvilangam K, Dhillon A, Saunders BP. Implementation of optical diagnosis with a "resect and discard" strategy in clinical practice: DISCARD3 study. Gastrointest Endosc. 2022;96:1021-1032.e2. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 1] [Cited by in F6Publishing: 1] [Article Influence: 0.5] [Reference Citation Analysis (0)] |
| 7. | Repici A, Hassan C, Radaelli F, Occhipinti P, De Angelis C, Romeo F, Paggi S, Saettone S, Cisarò F, Spaander M, Sharma P, Kuipers EJ. Accuracy of narrow-band imaging in predicting colonoscopy surveillance intervals and histology of distal diminutive polyps: results from a multicenter, prospective trial. Gastrointest Endosc. 2013;78:106-114. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 40] [Cited by in F6Publishing: 46] [Article Influence: 4.2] [Reference Citation Analysis (0)] |
| 8. | Rondonotti E, Hassan C, Andrealli A, Paggi S, Amato A, Scaramella L, Repici A, Radaelli F. Clinical Validation of BASIC Classification for the Resect and Discard Strategy for Diminutive Colorectal Polyps. Clin Gastroenterol Hepatol. 2020;18:2357-2365.e4. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 13] [Cited by in F6Publishing: 15] [Article Influence: 3.8] [Reference Citation Analysis (0)] |
| 9. | Kominami Y, Yoshida S, Tanaka S, Sanomura Y, Hirakawa T, Raytchev B, Tamaki T, Koide T, Kaneda K, Chayama K. Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointest Endosc. 2016;83:643-649. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 157] [Cited by in F6Publishing: 147] [Article Influence: 18.4] [Reference Citation Analysis (0)] |
| 10. | Vleugels JLA, Greuter MJE, Hazewinkel Y, Coupé VMH, Dekker E. Implementation of an optical diagnosis strategy saves costs and does not impair clinical outcomes of a fecal immunochemical test-based colorectal cancer screening program. Endosc Int Open. 2017;5:E1197-E1207. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 13] [Cited by in F6Publishing: 15] [Article Influence: 2.1] [Reference Citation Analysis (0)] |
| 11. | Weigt J, Repici A, Antonelli G, Afifi A, Kliegis L, Correale L, Hassan C, Neumann H. Performance of a new integrated computer-assisted system (CADe/CADx) for detection and characterization of colorectal neoplasia. Endoscopy. 2022;54:180-184. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 22] [Cited by in F6Publishing: 35] [Article Influence: 17.5] [Reference Citation Analysis (0)] |
| 12. | van der Zander QEW, Schreuder RM, Fonollà R, Scheeve T, van der Sommen F, Winkens B, Aepli P, Hayee B, Pischel AB, Stefanovic M, Subramaniam S, Bhandari P, de With PHN, Masclee AAM, Schoon EJ. Optical diagnosis of colorectal polyp images using a newly developed computer-aided diagnosis system (CADx) compared with intuitive optical diagnosis. Endoscopy. 2021;53:1219-1226. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 7] [Cited by in F6Publishing: 7] [Article Influence: 2.3] [Reference Citation Analysis (0)] |
| 13. | Kusters K, Scheeve T, Dehghani N, van der Zander QE, Schreuder R-M, Masclee AA, Schoon E, van der Sommen F, de With PH. Colorectal polyp classification using confidence-calibrated convolutional neural networks. SPIE. 2022;. [DOI] [Cited in This Article: ] |
| 14. | Fujifilm. CAD EYE Detection and Characterisation, 2020. Available from: https://www.fujifilm.eu/eu/cadeye. [Cited in This Article: ] |
| 15. | Bisschops R, Hassan C, Bhandari P, Coron E, Neumann H, Pech O, Correale L, Repici A. BASIC (BLI Adenoma Serrated International Classification) classification for colorectal polyp characterization with blue light imaging. Endoscopy. 2018;50:211-220. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 52] [Cited by in F6Publishing: 51] [Article Influence: 8.5] [Reference Citation Analysis (0)] |
| 16. | Sano Y, Tanaka S, Kudo SE, Saito S, Matsuda T, Wada Y, Fujii T, Ikematsu H, Uraoka T, Kobayashi N, Nakamura H, Hotta K, Horimatsu T, Sakamoto N, Fu KI, Tsuruta O, Kawano H, Kashida H, Takeuchi Y, Machida H, Kusaka T, Yoshida N, Hirata I, Terai T, Yamano HO, Kaneko K, Nakajima T, Sakamoto T, Yamaguchi Y, Tamai N, Nakano N, Hayashi N, Oka S, Iwatate M, Ishikawa H, Murakami Y, Yoshida S, Saito Y. Narrow-band imaging (NBI) magnifying endoscopic classification of colorectal tumors proposed by the Japan NBI Expert Team. Dig Endosc. 2016;28:526-533. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 294] [Cited by in F6Publishing: 316] [Article Influence: 39.5] [Reference Citation Analysis (1)] |
| 17. | Vinsard DG, Mori Y, Misawa M, Kudo SE, Rastogi A, Bagci U, Rex DK, Wallace MB. Quality assurance of computer-aided detection and diagnosis in colonoscopy. Gastrointest Endosc. 2019;90:55-63. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 92] [Cited by in F6Publishing: 81] [Article Influence: 16.2] [Reference Citation Analysis (0)] |
| 18. | de Groof AJ, Struyvenberg MR, Fockens KN, van der Putten J, van der Sommen F, Boers TG, Zinger S, Bisschops R, de With PH, Pouw RE, Curvers WL, Schoon EJ, Bergman JJGHM. Deep learning algorithm detection of Barrett's neoplasia with high accuracy during live endoscopic procedures: a pilot study (with video). Gastrointest Endosc. 2020;91:1242-1250. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 66] [Cited by in F6Publishing: 66] [Article Influence: 16.5] [Reference Citation Analysis (0)] |
| 19. | Rondonotti E, Hassan C, Tamanini G, Antonelli G, Andrisani G, Leonetti G, Paggi S, Amato A, Scardino G, Di Paolo D, Mandelli G, Lenoci N, Terreni N, Andrealli A, Maselli R, Spadaccini M, Galtieri PA, Correale L, Repici A, Di Matteo FM, Ambrosiani L, Filippi E, Sharma P, Radaelli F. Artificial intelligence-assisted optical diagnosis for the resect-and-discard strategy in clinical practice: the Artificial intelligence BLI Characterization (ABC) study. Endoscopy. 2023;55:14-22. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 7] [Cited by in F6Publishing: 33] [Article Influence: 33.0] [Reference Citation Analysis (0)] |
| 20. | Yoshida N, Inoue K, Tomita Y, Kobayashi R, Hashimoto H, Sugino S, Hirose R, Dohi O, Yasuda H, Morinaga Y, Inada Y, Murakami T, Zhu X, Itoh Y. An analysis about the function of a new artificial intelligence, CAD EYE with the lesion recognition and diagnosis for colorectal polyps in clinical practice. Int J Colorectal Dis. 2021;36:2237-2245. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 6] [Cited by in F6Publishing: 16] [Article Influence: 5.3] [Reference Citation Analysis (0)] |
| 21. | Hassan C, Sharma P, Mori Y, Bretthauer M, Rex DK; Combo Study Group, Repici A. Comparative Performance of Artificial Intelligence Optical Diagnosis Systems for Leaving in Situ Colorectal Polyps. Gastroenterology. 2023;164:467-469.e4. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 2] [Cited by in F6Publishing: 12] [Article Influence: 12.0] [Reference Citation Analysis (1)] |
| 22. | Jin EH, Lee D, Bae JH, Kang HY, Kwak MS, Seo JY, Yang JI, Yang SY, Lim SH, Yim JY, Lim JH, Chung GE, Chung SJ, Choi JM, Han YM, Kang SJ, Lee J, Chan Kim H, Kim JS. Improved Accuracy in Optical Diagnosis of Colorectal Polyps Using Convolutional Neural Networks with Visual Explanations. Gastroenterology. 2020;158:2169-2179.e8. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 59] [Cited by in F6Publishing: 74] [Article Influence: 18.5] [Reference Citation Analysis (0)] |
| 23. | Zachariah R, Samarasena J, Luba D, Duh E, Dao T, Requa J, Ninh A, Karnes W. Prediction of Polyp Pathology Using Convolutional Neural Networks Achieves "Resect and Discard" Thresholds. Am J Gastroenterol. 2020;115:138-144. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 52] [Cited by in F6Publishing: 73] [Article Influence: 18.3] [Reference Citation Analysis (0)] |
| 24. | Biffi C, Salvagnini P, Dinh NN, Hassan C, Sharma P; GI Genius CADx Study Group, Cherubini A. A novel AI device for real-time optical characterization of colorectal polyps. NPJ Digit Med. 2022;5:84. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 1] [Cited by in F6Publishing: 2] [Article Influence: 1.0] [Reference Citation Analysis (0)] |
| 25. | Sinonquel P, Eelbode T, Bossuyt P, Maes F, Bisschops R. Artificial intelligence and its impact on quality improvement in upper and lower gastrointestinal endoscopy. Dig Endosc. 2021;33:242-253. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 11] [Cited by in F6Publishing: 13] [Article Influence: 4.3] [Reference Citation Analysis (0)] |
| 26. | Messmann H, Bisschops R, Antonelli G, Libânio D, Sinonquel P, Abdelrahim M, Ahmad OF, Areia M, Bergman JJGHM, Bhandari P, Boskoski I, Dekker E, Domagk D, Ebigbo A, Eelbode T, Eliakim R, Häfner M, Haidry RJ, Jover R, Kaminski MF, Kuvaev R, Mori Y, Palazzo M, Repici A, Rondonotti E, Rutter MD, Saito Y, Sharma P, Spada C, Spadaccini M, Veitch A, Gralnek IM, Hassan C, Dinis-Ribeiro M. Expected value of artificial intelligence in gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy. 2022;54:1211-1231. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 44] [Cited by in F6Publishing: 34] [Article Influence: 17.0] [Reference Citation Analysis (0)] |
| 27. | Hassan C, Balsamo G, Lorenzetti R, Zullo A, Antonelli G. Artificial Intelligence Allows Leaving-In-Situ Colorectal Polyps. Clin Gastroenterol Hepatol. 2022;20:2505-2513.e4. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 6] [Cited by in F6Publishing: 30] [Article Influence: 15.0] [Reference Citation Analysis (0)] |
| 28. | Ponugoti PL, Cummings OW, Rex DK. Risk of cancer in small and diminutive colorectal polyps. Dig Liver Dis. 2017;49:34-37. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 67] [Cited by in F6Publishing: 89] [Article Influence: 12.7] [Reference Citation Analysis (0)] |
| 29. | Gupta N, Bansal A, Rao D, Early DS, Jonnalagadda S, Wani SB, Edmundowicz SA, Sharma P, Rastogi A. Prevalence of advanced histological features in diminutive and small colon polyps. Gastrointest Endosc. 2012;75:1022-1030. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 124] [Cited by in F6Publishing: 126] [Article Influence: 10.5] [Reference Citation Analysis (0)] |
| 30. | Willems P, Djinbachian R, Ditisheim S, Orkut S, Pohl H, Barkun A, Bouin M, Faulques B, von Renteln D. Uptake and barriers for implementation of the resect and discard strategy: an international survey. Endosc Int Open. 2020;8:E684-E692. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 18] [Cited by in F6Publishing: 11] [Article Influence: 2.8] [Reference Citation Analysis (0)] |
| 31. | Sánchez-Montes C, Sánchez FJ, Bernal J, Córdova H, López-Cerón M, Cuatrecasas M, Rodríguez de Miguel C, García-Rodríguez A, Garcés-Durán R, Pellisé M, Llach J, Fernández-Esparrach G. Computer-aided prediction of polyp histology on white light colonoscopy using surface pattern analysis. Endoscopy. 2019;51:261-265. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 43] [Cited by in F6Publishing: 48] [Article Influence: 9.6] [Reference Citation Analysis (0)] |
| 32. | Rodriguez-Diaz E, Baffy G, Lo WK, Mashimo H, Vidyarthi G, Mohapatra SS, Singh SK. Real-time artificial intelligence-based histologic classification of colorectal polyps with augmented visualization. Gastrointest Endosc. 2021;93:662-670. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 34] [Cited by in F6Publishing: 37] [Article Influence: 12.3] [Reference Citation Analysis (0)] |
| 33. | Renner J, Phlipsen H, Haller B, Navarro-Avila F, Saint-Hill-Febles Y, Mateus D, Ponchon T, Poszler A, Abdelhafez M, Schmid RM, von Delius S, Klare P. Optical classification of neoplastic colorectal polyps - a computer-assisted approach (the COACH study). Scand J Gastroenterol. 2018;53:1100-1106. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 28] [Cited by in F6Publishing: 32] [Article Influence: 5.3] [Reference Citation Analysis (0)] |
| 34. | Mori Y, Kudo SE, Misawa M, Saito Y, Ikematsu H, Hotta K, Ohtsuka K, Urushibara F, Kataoka S, Ogawa Y, Maeda Y, Takeda K, Nakamura H, Ichimasa K, Kudo T, Hayashi T, Wakamura K, Ishida F, Inoue H, Itoh H, Oda M, Mori K. Real-Time Use of Artificial Intelligence in Identification of Diminutive Polyps During Colonoscopy: A Prospective Study. Ann Intern Med. 2018;169:357-366. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 299] [Cited by in F6Publishing: 300] [Article Influence: 50.0] [Reference Citation Analysis (1)] |
| 35. | Leggett B, Whitehall V. Role of the serrated pathway in colorectal cancer pathogenesis. Gastroenterology. 2010;138:2088-2100. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 677] [Cited by in F6Publishing: 682] [Article Influence: 48.7] [Reference Citation Analysis (1)] |
| 36. | Young J, Jenkins M, Parry S, Young B, Nancarrow D, English D, Giles G, Jass J. Serrated pathway colorectal cancer in the population: genetic consideration. Gut. 2007;56:1453-1459. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 71] [Cited by in F6Publishing: 75] [Article Influence: 4.4] [Reference Citation Analysis (0)] |