Baishideng Publishing Group Inc, 7041 Koll Center Parkway, Suite 160, Pleasanton, CA 94566, USA
Share the Article
He HW, Chan IN, Qin YY, Yan T, Wong PK. Multimodal artificial intelligence and large language models in gastrointestinal endoscopy: From lesion detection to workflow-integrated report generation. World J Gastrointest Endosc 2026; 18(11): 126153 [DOI: 10.4253/wjge.126153]
World J Gastrointest Endosc. Nov 16, 2026; 18(11): 126153 Published online Nov 16, 2026. doi: 10.4253/wjge.126153
Multimodal artificial intelligence and large language models in gastrointestinal endoscopy: From lesion detection to workflow-integrated report generation
Hua-Wei He, In Neng Chan, Ye-Ying Qin, Tao Yan, Pak Kin Wong
Hua-Wei He, Tao Yan, School of Mechanical Engineering, Hubei University of Arts and Science, Xiangyang 441053, Hubei Province, China
In Neng Chan, Pak Kin Wong, Department of Biomedical Engineering, University of Macau, Macau 999078, China
Ye-Ying Qin, Pak Kin Wong, Department of Electromechanical Engineering, University of Macau, Macau 999078, China
Co-first authors: Hua-Wei He and In Neng Chan.
Co-corresponding authors: Tao Yan and Pak Kin Wong.
Author contributions: He HW and Yan T conceptualized the study, contributed to methodology and resources, were responsible for writing the original draft; Yan T and Wong PK provided supervision, project administration, and funding acquisition, played important and indispensable roles in the manuscript preparation as the co-corresponding authors; Qin YY conducted the investigation and contributed to writing, review, and editing; Chan IN, Yan T and Wong PK critically revised the manuscript; He HW and Chan IN made crucial contributions to the completion of the project and therefore qualify as co-first authors; all authors read and approved the final version of the manuscript.
AI contribution statement: Portions of this manuscript were edited using AI tools solely for language refinement. The authors carefully reviewed and verified all AI-assisted outputs and take full responsibility for the scientific content of the manuscript.
Supported by the Guangdong Basic and Applied Basic Research Foundation, No. 2021B1515130003; the Key Research and Development Plan of Hubei Province, No. 2022BCE034; and the Natural Science Foundation of Hubei Province, No. 2024AFB1054.
Conflict-of-interest statement: All authors declare no conflict of interest.
Corresponding author: Pak Kin Wong, PhD, Professor, Department of Biomedical Engineering, University of Macau, Avenida da Universidade, Taipa, Macau 999078, China. fstpkw@um.edu.mo
Received: August 3, 2026 Revised: September 17, 2026 Accepted: October 21, 2026 Published online: November 16, 2026 Processing time: 109 Days and 21.4 Hours
Core Tip
Core Tip: Artificial intelligence in gastrointestinal endoscopy is evolving from single lesion identification and diagnosis to multimodal, workflow-oriented support. The review explores methods to integrate endoscopic images and videos with clinical text, histology, and procedural data using multimodal learning, vision-language models, multimodal large language models (LLMs), and complementary text-based LLMs. The maturity of the clinical evidence varies greatly; however, the current evidence supports applications in lesion classification, procedural quality assessment, systematic documentation and draft report generation. Limited multimodal datasets, hallucination and grounding issues, lack of external validation, workflow integration, privacy and regulatory restrictions are still key challenges for mainstream clinical application.
