Artificial intelligence in hematopoietic stem cell research and associated malignancies: From disease modeling to cell manufacturing

Table 5 Validation checklist for leukemic stem cell/hematopoietic stem cell artificial intelligence models, adapted from TRIPOD + AI and PROBAST + AI guidelines with hematopoietic stem cell/Leukemic stem cell-specific requirements

Validation domain	Key requirement	LSC/HSC-specific considerations
Cohort representativeness	The training cohort must represent the target clinical population with respect to age, disease stage, and treatment era	LSC/HSC models should include balanced representation of ELN risk categories, stem-cell compartment measurements (CD34+CD38- frequencies), and both newly diagnosed and relapsed/refractory patients[52,106-108]
Event counts	An adequate number of outcome events (relapse, death, MRD positivity) to prevent overfitting	Minimum 10-20 events per predictor variable; for LSC-specific endpoints (e.g., LSC+ vs LSC-), ensure sufficient LSC+ cases across validation sets[52,55]
Internal validation	Model performance assessed on held-out data from the same source (cross-validation or hold-out split)	Report performance metrics (AUROC, calibration) separately for LSC-enriched vs LSC-depleted subgroups if the model claims to encode stemness biology[52,109,110]
External validation	Independent cohort from a different institution, time period, or geography	Essential for LSC models given center-to-center variability in LSC phenotyping protocols and MRD detection thresholds[52,106,110]
Prospective evaluation	Forward-looking validation on newly enrolled patients before clinical deployment	Required for LSC-targeted therapy selection models to confirm that AI predictions align with clinical outcomes under prospective conditions[55,109,111]
Dataset shift detection	Assess whether model performance degrades when applied to data with distributional differences (batch effects, assay drift)	Critical for flow cytometry-based LSC models: Validate across different antibody panels, fluorophores, and cytometers; report performance stratified by batch[55,112]
Calibration assessment	Predicted probabilities should match observed event frequencies	For LSC burden models, calibration plots should show agreement between predicted LSC frequency (or surrogate score) and directly measured LSC% by flow cytometry in the calibration subset[52,55,106]
Decision-curve analysis	Net benefit of model-guided decisions compared to treat-all or treat-none strategies	For LSC-directed therapies (venetoclax, Menin inhibitors), decision curves should quantify clinical utility across risk thresholds relevant to treatment intensification decisions[55,111]
Explainability and feature attribution	Use of XAI methods (SHAP, attention weights) to identify which features drive predictions	Essential for validating LSC-AI hypothesis: Determine whether high-risk predictions are driven by known stemness genes (17-gene LSC score, HOXMEIS1 programs) or alternative pathways[52,113]
Bias and fairness evaluation	Assess performance stratified by demographic subgroups and underrepresented populations	Evaluate whether LSC models perform equivalently across age groups (pediatric vs adult vs elderly AML), ancestry, and sex; report subgroup-specific metrics[107]
Missing data handling	Transparent reporting of missingness patterns and imputation strategies	LSC models often integrate multi-omics data with heterogeneous completeness (e.g., scRNA-seq available for a subset); clearly document handling of missing modalities and proteins in CITE-seq[111]
Comparator benchmarking	Performance compared to established clinical risk systems	For AML LSC models, benchmark against ELN 2022 risk classification, 17-gene LSC score, and LSC frequency by flow cytometry; report incremental predictive value[114]

LSC: Leukemic stem cell; HSC: Hematopoietic stem cell; ELN: European Leukemia Net; MRD: Measurable residual disease; AUROC: Area under the receiver operating characteristic; AI: Artificial intelligence; XAI: Explainable artificial intelligence; SHAP: SHapley Additive exPlanations; AML: Acute myeloid leukemia; scRNA-seq: Single-cell RNA sequencing; CITE-seq: Cellular indexing of transcriptomes and epitopes by sequencing.