Multimodal artificial intelligence and large language models in gastrointestinal endoscopy: From lesion detection to workflow-integrated report generation

Table 1 Distinction among multimodal artificial intelligence concepts in gastrointestinal endoscopy

Concept	Direct model input	Key distinction	Typical outputs	Key limitations
Multimodal learning	≥ 2 heterogeneous data sources	Broad fusion/alignment paradigm; not a specific architecture	Prediction, cross-modal representation, decision support, structured output	Missing and asynchronous data
VLM	Image/video + text	Learns visual–language representations or alignment	Image-text alignment, retrieval, annotation, visual description	Weak temporal modeling and limited grounding
MLLM	Visual + text + multimodal input	LLM-centered generation conditioned on visual input	Visual question answering, draft report generation	Hallucination and long-video challenges
Text-based LLM	Text	No direct access to endoscopic visual input	Summarization, information extraction, patient communication	Inability to directly interpret images

VLM: Vision-language model; MLLM: Multimodal large language model; LLM: Large language model.