Domain specific training with three-dimensional virtual colonoscopy images improves artificial intelligence based automated size prediction for large lesions

Figure 1 Violin plot displaying ground truth lesion size distribution of the endoscopic submucosal dissection and the external dataset. Average lesion size was much smaller in the external dataset compared to the endoscopic submucosal dissection set. ESD: Endoscopic submucosal dissection.

Figure 2 Visual inspection indicates that all models generate plausible representations of the depth structure. A: The first row shows an image from the virtual colonoscopy alongside the predicted depth map of the fine-tuned Depth Anything V2 model. Depth is encoded in grayscale, with black indicating greater distance and white indicating closer proximity to the camera; B: The second row presents a representative whitelight endoscopic image from the endoscopic submucosal dissection cohort, accompanied by depth map predictions generated by Apple Depth Pro, UniK3D, Depth Anything V2 (nonfinetuned), as well as Depth Anything V2 (fine-tuned). Qualitatively, all models provide a reasonable representation of the depth structure in the image; C: The third row illustrates a representative whitelight endoscopic image from the external dataset, together with the corresponding depth map predictions from Apple Depth Pro, UniK3D, Depth Anything V2 (non-fine-tuned), and Depth Anything V2 (fine-tuned).

Figure 3 Most prediction errors lie within the limits of agreement, suggesting that the model’s measurements are largely consistent with the gold-standard caliper measurements. A: In the endoscopic submucosal dissection cohort, the first plot illustrates the mean absolute percentage error across the three baseline models and the finetuned model. The best AI model, DphAnyV2_3DVCol, showed comparable mean percentage deviation as the trainees, while the experts still did more accurate predictions. The second plot depicts the prediction of the fine-tuned Depth Anything V2 model (DphAnyV2_3DVCol) and the experts prediction compared with the ground-truth size. Model and Experts show a similar error distribution. The third plot presents the Bland-Altman analysis for the best fine-tuned Depth Anything V2 model. The majority of this models predictions lie within the limits of agreement, suggesting good agreement of with the gold-standard caliper measurements; B: Results from the external dataset. The first graph shows the mean absolute percentage errors for the three baseline models and the fine-tuned model. The second plot depicts the prediction of the finetuned Depth Anything V2 model against the ground-truth size. Smaller polyp sizes lead to less outliers than larger sizes. The third plot presents the Bland-Altman analysis for the best fine-tuned Depth Anything V2 model.

Figure 4 Percentage of correctly binned size predictions per size category for four artificial intelligence models (AplDepthPro, UniK3D, DphAnyV2, DphAnyV2_3DVCol). A: Percentage of correctly binned size predictions per size category for four artificial intelligence (AI) models (AplDphPro, UniK3D, DphAnyV2, DphAnyV2_3DVCol), pooled experts, and pooled trainees. Measurements were classified into five bins (≤ 5 mm, 6-9 mm, 10-19 mm, 20-39 mm, > 39 mm). Across the three well-populated bins, experts showed the most consistent accuracy (49%-68%), while trainees performed well for smaller sizes (Bin 3: 64%) but markedly declined for larger measurements (Bin 5: 10%). Among AI models, DphAnyV2_3DVCol achieved the highest accuracy in the larger size ranges; B: Percentage of correctly binned size predictions per size category for four AI models (AplDepthPro, UniK3D, DphAnyV2, DphAnyV2_3DVCol) in a dataset restricted to smaller measurements. All models performed best in Bin 3, with three of four reaching 75% accuracy. Performance was notably weaker in Bin 2 across all models, with AplDepthPro achieving the lowest accuracy (17%).