Multimodal artificial intelligence in capsule endoscopy: Integrating video and sensor data for advanced gastrointestinal diagnostics

Table 2 Comparative overview of image-only vs multimodal artificial intelligence architectures in capsule endoscopy¹

Model	Modality type	Capsule platform	Key diagnostic application	Performance accuracy
Ding et al[27], 2023 (CNN)	Image only	Small bowel capsule	Lesion detection (ulcers, bleeding)	Sensitivity 99.2%, specificity 96.7%
Nam et al[11], 2024 (CNN-LSTM)	Multimodal (video + IMU)	PillCam™ SB3	Organ localization and transit time estimation	> 95% accuracy
Turan et al[31], 2017 (Endo-VMFuseNet)	Multimodal (video + magnetic)	Experimental platform	Capsule localization (3D trajectory mapping)	Sub-millimeter accuracy
Mascarenhas et al[42], 2024 (CNN)	Image only	Multibrand CE	Vascular lesion detection	Accuracy 95%, sensitivity 86.4%, specificity 98.3%
Vedaei and Wahid[13], 2021 (prototype)	Multimodal (video + IMU)	Research prototype	3D trajectory reconstruction	Improved localization accuracy

¹Summary of representative studies evaluating artificial intelligence (AI) systems in capsule endoscopy, comparing image-only convolutional neural network (CNN) models with multimodal architectures that integrate additional sensor inputs such as inertial measurement units or magnetic trackers. Image-only CNN models show excellent lesion detection accuracy (95%-99%) for bleeding and ulcer identification, whereas multimodal systems, such as CNN-long short-term memory and Endo-VMFuseNet, achieve superior spatial localization and temporal mapping capabilities (> 95% accuracy, sub-millimeter precision). These findings highlight that while image-only AI excels in visual lesion recognition multimodal approaches provide enhanced anatomical context and diagnostic robustness.

LSTM: Long short-term memory network; 3D: Three-dimensional; CNN: Convolutional neural networks; IMU: Inertial measurement units; CE: Capsule endoscopy.