Multimodal artificial intelligence integrates imaging, endoscopic, and omics data for intelligent decision-making in individualized gastrointestinal tumor treatment

Table 4 Challenges and future directions of multimodal artificial intelligence in gastrointestinal cancer therapy

Core challenges	Key technologies/methods	Future directions
Data quality & privacy protection: Data heterogeneity (divergent formats/standards); data noise (equipment/operator variations). Patient privacy risks (esp. genomic/imaging data)	Data standardization: Common data models (e.g., OMOP CDM, medical imaging CDM); Privacy-preserving techniques: FL, DP, Blockchain; Legal compliance: Frameworks like GDPR to enhance policy transparency	To build a more secure and reliable data environment, promoting seamless integration and controlled sharing of high-quality data
Model interpretability & clinical acceptability: "Black-box" problem erodes clinical trust. Opaque decision-making hinders regulatory approval & integration	Explainable AI: Attention mechanisms, prototype networks (ProtoPNet), Counterfactual explanations; Interpretability tools: LIME, SHAP, Grad-CAM for visualization & feature importance ranking; Clinical integration: Displaying model uncertainty & key decision factors in CDSS	To develop transparent and trustworthy AI systems, enhance clinician trust, and promote deep integration of AI into clinical workflows
Multi-center collaboration & standardization: Significant data heterogeneity across centers (equipment, protocols, populations). Poor model generalizability, hindering cross-institutional application	Multi-center data sharing & standardization: Unified data formats and acquisition standards; privacy-preserving collaborative training: Federated learning for joint modeling; standardized multimodal databases: Integrating genomics, radiomics, and other multidimensional data	To promote large-scale, high-quality multi-center collaboration, establish industry standards, and improve model generalizability and clinical applicability
Technical integration & clinical translation: Reliance on large annotated datasets limits generalizability. Barriers in translating research findings to clinical application	Emerging ML paradigms: RL for dynamic treatment optimization; SSL to reduce annotation dependency; Integrating Novel Data types: e.g., digital pathology, patient behavior data; Robust clinical validation: Validating model efficacy and robustness through clinical trials and RWD	To integrate multimodal AI with cutting-edge technologies and validate it through rigorous clinical trials, ultimately enabling its routine use in personalized therapy

AI: Artificial intelligence; FL: Federated learning; DP: Differential privacy; RL: Reinforcement learning; SSL: Self-Supervised learning; RWD: Real-world data; CDM; Common data model; OMOP: Observational medical outcomes partnership; GDPR: General data protection regulation; LIME: Local interpretable model-agnostic explanations; SHAP: SHapley Additive exPlanations; Grad-CAM: Gradient-weighted Class Activation Mapping; CDSS: Clinical decision support systems.