1
|
El Moheb M, Shen C, Kim S, Cummins K, Sears O, Sahli Z, Zhang H, Hedrick T, Witt RG, Tsung A. A novel artificial intelligence framework to quantify the impact of clinical compared with nonclinical influences on postoperative length of stay. Surgery 2025; 181:109152. [PMID: 39891965 DOI: 10.1016/j.surg.2025.109152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2024] [Revised: 12/20/2024] [Accepted: 01/05/2025] [Indexed: 02/03/2025]
Abstract
BACKGROUND The relative proportion of clinical compared with nonclinical influences on length of stay after colectomy has never been measured. We developed a novel machine-learning framework that quantifies the proportion of length of stay after colectomy attributable to clinical factors and infers the overall impact of nonclinical influences. STUDY DESIGN Patients who underwent partial colectomy, total colectomy, or low anterior resection included in American College of Surgeons National Surgical Quality Improvement were analyzed. Multivariable linear regression, random forest, and neural network models were developed to assess the impact of 56 clinical variables on length of stay. The random forest and neural network models were fine-tuned to maximize the explanatory power of clinical variables on length of stay. R2 measured the proportion of length of stay explained by clinical factors. The contribution of nonclinical factors was inferred from residual analysis. Mean absolute error was used to measure the discrepancy between actual and model-predicted length of stay. RESULTS Of 96,081 patients, 71% underwent partial colectomy (mean length of stay, 6.8 days; standard deviation, 5.6), 27% low anterior resection (5.4; 4.4), and 2% total colectomy (11.8; 7.1). Clinical factors in multivariable linear regression models accounted for only 29-54% of length of stay variability. The random forest and neural network models demonstrated persistent unexplained length of stay variability even when considering nonlinear interactions (R2: random forest [range, 0.46-0.55]; neural network [range, 0.44-0.57]), consistent with multivariable linear regression models. Mean absolute error showed clinical factors could not account for 2-2.5 days of length of stay after low anterior resection and partial colectomy, and 4 days after total colectomy. CONCLUSION This is the first study to quantify the overall influence of clinical factors on post-colectomy length of stay, revealing they explain less than 55% of variability. By maximizing clinical factors' explanatory impact using machine learning, the remaining variability is inferred to be nonclinical. Our findings provide hospitals with a novel paradigm to indirectly measure the influence of previously elusive nonclinical factors.
Collapse
Affiliation(s)
- Mohamad El Moheb
- Department of Surgery, University of Virginia, Charlottesville, VA; School of Data Science, University of Virginia, Charlottesville, VA.
| | - Chengli Shen
- Department of Surgery, University of Virginia, Charlottesville, VA
| | - Susan Kim
- Department of Surgery, University of Virginia, Charlottesville, VA
| | - Kaelyn Cummins
- Department of Surgery, University of Virginia, Charlottesville, VA
| | - Olivia Sears
- Department of Surgery, University of Virginia, Charlottesville, VA
| | - Zeyad Sahli
- Department of Surgery, University of Virginia, Charlottesville, VA
| | - Hongji Zhang
- Department of Surgery, University of Virginia, Charlottesville, VA
| | - Traci Hedrick
- Department of Surgery, University of Virginia, Charlottesville, VA. https://twitter.com/tlhedr0
| | - Russell G Witt
- Department of Surgery, University of Virginia, Charlottesville, VA
| | - Allan Tsung
- Department of Surgery, University of Virginia, Charlottesville, VA. https://twitter.com/allantsung
| |
Collapse
|
2
|
Ducas A, Mangano A, Borgioli L, Cassiani J, Lopez P, Giulianotti PC. The role of artificial intelligence in pancreatic surgery: Current and future perspectives. Ann Hepatobiliary Pancreat Surg 2025; 29:1-4. [PMID: 39228175 PMCID: PMC11830889 DOI: 10.14701/ahbps.24-130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/25/2024] [Accepted: 07/31/2024] [Indexed: 09/05/2024] Open
Affiliation(s)
- Alvaro Ducas
- Division of General, Minimally Invasive and Robotic Surgery, Department of Surgery, University of Illinois at Chicago, Chicago, IL, United States
| | - Alberto Mangano
- Division of General, Minimally Invasive and Robotic Surgery, Department of Surgery, University of Illinois at Chicago, Chicago, IL, United States
| | - Leonardo Borgioli
- Department of Electrical and Computer Engineering, College of Engineering, University of Illinois at Chicago, Chicago, IL, United States
| | - Jessica Cassiani
- Division of General, Minimally Invasive and Robotic Surgery, Department of Surgery, University of Illinois at Chicago, Chicago, IL, United States
| | - Paula Lopez
- Division of General, Minimally Invasive and Robotic Surgery, Department of Surgery, University of Illinois at Chicago, Chicago, IL, United States
| | - Pier Cristoforo Giulianotti
- Division of General, Minimally Invasive and Robotic Surgery, Department of Surgery, University of Illinois at Chicago, Chicago, IL, United States
| |
Collapse
|
3
|
Catalano G, Alaimo L, Endo Y, Chatzipanagiotou OP, Ruzzenente A, Aldrighetti L, Weiss M, Bauer TW, Alexandrescu S, Poultsides GA, Maithel SK, Marques HP, Martel G, Pulitano C, Shen F, Cauchy F, Koerkamp BG, Endo I, Kitago M, Pawlik TM. Development and Validation of a Predictive Risk Score for Blood Transfusion in Patients Undergoing Curative-Intent Surgery for Intrahepatic Cholangiocarcinoma. J Surg Oncol 2025; 131:242-251. [PMID: 39285653 PMCID: PMC12035667 DOI: 10.1002/jso.27903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2024] [Accepted: 08/30/2024] [Indexed: 04/29/2025]
Abstract
BACKGROUND AND OBJECTIVES Among patients undergoing liver resection for intrahepatic cholangiocarcinoma (ICC), perioperative bleeding requiring blood transfusion is a common complication, yet preoperative identification of patients at risk for transfusion remains challenging. The objective of this study was to develop a preoperative risk score for blood transfusion requirement during surgery for ICC. METHODS Patients undergoing curative-intent liver surgery for ICC (1990-2020) were identified from a multi-institutional database. A predictive model was developed and validated. An easy-to-use risk calculator was made available online. RESULTS Among 1420 patients, 300 (21.1%) received an intraoperative transfusion. Independent predictors of transfusion included severe preoperative anemia (OR = 1.65, 95% CI 1.10-2.47), T2 category or higher (OR = 2.00, 95% CI 1.36-3.02), positive lymph nodes (OR = 1.75, 95% CI 1.32-2.32) and major resection (OR = 2.56, 95%CI 1.85-3.58). Receipt of blood transfusion significantly correlated with worse outcomes. The model showed good discriminative ability in both training (AUC = 0.68, 95% CI 0.66-0.72) and bootstrapping validation (C-index = 0.67, 95% CI 0.65-0.70) cohorts. An online risk calculator of blood transfusion requirement was developed (https://catalano-giovanni.shinyapps.io/TransfusionRisk). CONCLUSIONS Intraoperative blood transfusion was significantly associated with poor postoperative outcomes among patients undergoing surgery for ICC. The identification of patients at high risk of transfusion could improve perioperative patient care and blood resources allocation.
Collapse
Affiliation(s)
- Giovanni Catalano
- Department of SurgeryThe Ohio State University Wexner Medical Center and James Comprehensive Cancer CenterColumbusOhioUSA
- Department of SurgeryUniversity of VeronaVeronaItaly
| | - Laura Alaimo
- Department of SurgeryThe Ohio State University Wexner Medical Center and James Comprehensive Cancer CenterColumbusOhioUSA
- Department of SurgeryUniversity of VeronaVeronaItaly
| | - Yutaka Endo
- Department of SurgeryThe Ohio State University Wexner Medical Center and James Comprehensive Cancer CenterColumbusOhioUSA
| | - Odysseas P. Chatzipanagiotou
- Department of SurgeryThe Ohio State University Wexner Medical Center and James Comprehensive Cancer CenterColumbusOhioUSA
| | | | | | - Matthew Weiss
- Department of SurgeryJohns Hopkins HospitalBaltimoreMarylandUSA
| | - Todd W. Bauer
- Department of SurgeryUniversity of VirginiaCharlottesvilleVirginiaUSA
| | | | | | | | | | | | - Carlo Pulitano
- Department of Surgery, Royal Prince Alfred HospitalUniversity of SydneySydneyNew South WalesAustralia
| | - Feng Shen
- Department of SurgeryEastern Hepatobiliary Surgery HospitalShanghaiChina
| | - François Cauchy
- Department of Hepatobiliopancreatic Surgery and Liver Transplantation, AP‐HPBeaujon HospitalClichyFrance
| | - Bas G. Koerkamp
- Department of SurgeryErasmus University Medical CentreRotterdamThe Netherlands
| | - Itaru Endo
- Department of Gastroenterological SurgeryYokohama City University School of MedicineYokohamaJapan
| | | | - Timothy M. Pawlik
- Department of SurgeryThe Ohio State University Wexner Medical Center and James Comprehensive Cancer CenterColumbusOhioUSA
| |
Collapse
|
4
|
Dichter A, Bhatt K, Liu M, Park T, Djalilian HR, Abouzari M. Post-Operative Outcome Predictions in Vestibular Schwannoma Using Machine Learning Algorithms. J Pers Med 2024; 14:1170. [PMID: 39728082 PMCID: PMC11678011 DOI: 10.3390/jpm14121170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2024] [Revised: 12/16/2024] [Accepted: 12/20/2024] [Indexed: 12/28/2024] Open
Abstract
Background/Objectives: This study aimed to develop a machine learning (ML) algorithm that can predict unplanned reoperations and surgical/medical complications after vestibular schwannoma (VS) surgery. Methods: All pre- and peri-operative variables available in the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) database (n = 110), except those directly related to our outcome variables, were used as input variables. A deep neural network model consisting of seven layers was developed using the Keras open-source library, with a 70:30 breakdown for training and testing. The feature importance of input variables was measured to elucidate their relative permutation effect in the ML model. Results: Of the 1783 patients with VS undergoing surgery, unplanned reoperation, surgical complications, and medical complications were seen in 8.5%, 5.2%, and 6.2% of patients, respectively. The deep neural network model had area under the curve of receiver operating characteristics (ROC-AUC) of 0.6315 (reoperation), 0.7939 (medical complications), and 0.719 (surgical complications). Accuracy, specificity, and negative predictive values of the model for all outcome variables ranged from 82.1 to 96.6%, while positive predictive values and sensitivity ranged from 16.7 to 51.5%. Variables such as the length of stay post-operation until discharge, days from operation to discharge, and the total hospital length of stay had the highest permutation importance. Conclusions: We developed an effective ML algorithm predicting unplanned reoperation and surgical/medical complications post-VS surgery. This may offer physicians guidance into potential post-surgical outcomes to allow for personalized medical care plans for VS patients.
Collapse
Affiliation(s)
| | | | | | | | | | - Mehdi Abouzari
- Division of Neurotology and Skull Base Surgery, Department of Otolaryngology-Head and Neck Surgery, University of California, Irvine, CA 92697, USA
| |
Collapse
|
5
|
van Boekel AM, van der Meijden SL, Arbous SM, Nelissen RGHH, Veldkamp KE, Nieswaag EB, Jochems KFT, Holtz J, Veenstra AVIJ, Reijman J, de Jong Y, van Goor H, Wiewel MA, Schoones JW, Geerts BF, de Boer MGJ. Systematic evaluation of machine learning models for postoperative surgical site infection prediction. PLoS One 2024; 19:e0312968. [PMID: 39666725 PMCID: PMC11637340 DOI: 10.1371/journal.pone.0312968] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2024] [Accepted: 10/15/2024] [Indexed: 12/14/2024] Open
Abstract
BACKGROUND Surgical site infections (SSIs) lead to increased mortality and morbidity, as well as increased healthcare costs. Multiple models for the prediction of this serious surgical complication have been developed, with an increasing use of machine learning (ML) tools. OBJECTIVE The aim of this systematic review was to assess the performance as well as the methodological quality of validated ML models for the prediction of SSIs. METHODS A systematic search in PubMed, Embase and the Cochrane library was performed from inception until July 2023. Exclusion criteria were the absence of reported model validation, SSIs as part of a composite adverse outcome, and pediatric populations. ML performance measures were evaluated, and ML performances were compared to regression-based methods for studies that reported both methods. Risk of bias (ROB) of the studies was assessed using the Prediction model Risk of Bias Assessment Tool. RESULTS Of the 4,377 studies screened, 24 were included in this review, describing 85 ML models. Most models were only internally validated (81%). The C-statistic was the most used performance measure (reported in 96% of the studies) and only two studies reported calibration metrics. A total of 116 different predictors were described, of which age, steroid use, sex, diabetes, and smoking were most frequently (100% to 75%) incorporated. Thirteen studies compared ML models to regression-based models and showed a similar performance of both modelling methods. For all included studies, the overall ROB was high or unclear. CONCLUSIONS A multitude of ML models for the prediction of SSIs are available, with large variability in performance. However, most models lacked external validation, performance was reported limitedly, and the risk of bias was high. In studies describing both ML models and regression-based models, one modelling method did not outperform the other.
Collapse
Affiliation(s)
- Anna M. van Boekel
- Department of Internal Medicine, Leiden University Medical Center, Leiden, The Netherlands
| | - Siri L. van der Meijden
- Department of Intensive Care, Leiden University Medical Center, Leiden, The Netherlands
- Healthplus.ai R&D B.V., Amsterdam, The Netherlands
| | - Sesmu M. Arbous
- Department of Intensive Care, Leiden University Medical Center, Leiden, The Netherlands
| | - Rob G. H. H. Nelissen
- Department of Orthopedic surgery, Leiden University Medical Center, Leiden, The Netherlands
| | - Karin E. Veldkamp
- Department of Medical Microbiology and Infection Control, Leiden University Medical Center, Leiden, The Netherlands
| | - Emma B. Nieswaag
- Department of Intensive Care, Leiden University Medical Center, Leiden, The Netherlands
- Healthplus.ai R&D B.V., Amsterdam, The Netherlands
| | - Kim F. T. Jochems
- Department of Intensive Care, Leiden University Medical Center, Leiden, The Netherlands
- Healthplus.ai R&D B.V., Amsterdam, The Netherlands
| | - Jeroen Holtz
- Department of Intensive Care, Leiden University Medical Center, Leiden, The Netherlands
- Healthplus.ai R&D B.V., Amsterdam, The Netherlands
| | - Annekee van IJlzinga Veenstra
- Department of Intensive Care, Leiden University Medical Center, Leiden, The Netherlands
- Healthplus.ai R&D B.V., Amsterdam, The Netherlands
| | - Jeroen Reijman
- Department of Intensive Care, Leiden University Medical Center, Leiden, The Netherlands
- Healthplus.ai R&D B.V., Amsterdam, The Netherlands
| | - Ype de Jong
- Department of Internal Medicine, Leiden University Medical Center, Leiden, The Netherlands
- Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands
| | - Harry van Goor
- Department of Surgery, Radboud UMC, Nijmegen, The Netherlands
| | | | - Jan W. Schoones
- Waleus Medical Library, Leiden University Medical Center, Leiden, The Netherlands
| | | | - Mark G. J. de Boer
- Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands
- Department of Infectious disease, Leiden University Medical Center, Leiden, The Netherlands
| |
Collapse
|
6
|
Kim SH, Park SY, Seo H, Woo J. Feature selection integrating Shapley values and mutual information in reinforcement learning: An application in the prediction of post-operative outcomes in patients with end-stage renal disease. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 257:108416. [PMID: 39342877 DOI: 10.1016/j.cmpb.2024.108416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2024] [Revised: 08/28/2024] [Accepted: 09/06/2024] [Indexed: 10/01/2024]
Abstract
BACKGROUND In predicting post-operative outcomes for patients with end-stage renal disease, our study faced challenges related to class imbalance and a high-dimensional feature space. Therefore, with a focus on overcoming class imbalance and improving interpretability, we propose a novel feature selection approach using multi-agent reinforcement learning. METHODS We proposed a multi-agent feature selection model based on a comprehensive reward function that combines classification model performance, Shapley additive explanations values, and the mutual information. The definition of rewards in reinforcement learning is crucial for model convergence and performance improvement. Initially, we set a deterministic reward based on the mutual information between variables and the target class, selecting variables that are highly dependent on the class, thus accelerating convergence. We then prioritized variables that influence the minority class on a sample basis and introduced a dynamic reward distribution strategy using Shapley additive explanations values to improve interpretability and solve the class imbalance problem. RESULTS Involving the integration of electronic medical records, anesthesia records, operating room vital signs, and pre-operative anesthesia evaluations, our approach effectively mitigated class imbalance and demonstrated superior performance in ablation analysis. Our model achieved a 16% increase in the minority class F1 score and an 8.2% increase in the overall F1 score compared to the baseline model without feature selection. CONCLUSION This study contributes important research findings that show that the multi-agent-based feature selection method can be a promising approach for solving the class imbalance problem.
Collapse
Affiliation(s)
- Seo-Hee Kim
- Soonchunhyang University, Department of ICT Convergence, Asan, 31538, Republic of Korea
| | - Sun Young Park
- Soonchunhyang University Seoul Hospital, Anesthesiology and Pain Medicine, Seoul, 04401, Republic of Korea.
| | - Hyungseok Seo
- Kyung Hee University Hospital at Gangdong, Department of Anesthesiology and Pain Medicine, College of Medicine, Seoul, 05278, Republic of Korea
| | - Jiyoung Woo
- Soonchunhyang University, Department of AI and Big Data, Asan, 31538, Republic of Korea.
| |
Collapse
|
7
|
Hossain I, Madani A, Laplante S. Machine learning perioperative applications in visceral surgery: a narrative review. Front Surg 2024; 11:1493779. [PMID: 39539511 PMCID: PMC11557547 DOI: 10.3389/fsurg.2024.1493779] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2024] [Accepted: 10/18/2024] [Indexed: 11/16/2024] Open
Abstract
Artificial intelligence in surgery has seen an expansive rise in research and clinical implementation in recent years, with many of the models being driven by machine learning. In the preoperative setting, machine learning models have been utilized to guide indications for surgery, appropriate timing of operations, calculation of risks and prognostication, along with improving estimations of time and resources required for surgeries. Intraoperative applications that have been demonstrated are visual annotations of the surgical field, automated classification of surgical phases and prediction of intraoperative patient decompensation. Postoperative applications have been studied the most, with most efforts put towards prediction of postoperative complications, recurrence patterns of malignancy, enhanced surgical education and assessment of surgical skill. Challenges to implementation of these models in clinical practice include the need for more quantity and quality of standardized data to improve model performance, sufficient resources and infrastructure to train and use machine learning, along with addressing ethical and patient acceptance considerations.
Collapse
Affiliation(s)
- Intekhab Hossain
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada
| | - Amin Madani
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada
| | - Simon Laplante
- Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada
- Department of Surgery, Mayo Clinic, Rochester, MN, United States
| |
Collapse
|
8
|
Agnes A, Nguyen ST, Konishi T, Peacock O, Bednarski BK, You YN, Messick CA, Tillman MM, Skibber JM, Chang GJ, Uppal A. Early Postoperative Prediction of Complications and Readmission After Colorectal Cancer Surgery Using an Artificial Neural Network. Dis Colon Rectum 2024; 67:1341-1352. [PMID: 38959458 DOI: 10.1097/dcr.0000000000003253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 07/05/2024]
Abstract
BACKGROUND Early predictors of postoperative complications can risk-stratify patients undergoing colorectal cancer surgery. However, conventional regression models have limited power to identify complex nonlinear relationships among a large set of variables. We developed artificial neural network models to optimize the prediction of major postoperative complications and risk of readmission in patients undergoing colorectal cancer surgery. OBJECTIVE This study aimed to develop an artificial neural network model to predict postoperative complications using postoperative laboratory values and compare the accuracy of models to standard regression methods. DESIGN This retrospective study included patients who underwent elective colorectal cancer resection between January 1, 2016, and July 31, 2021. Clinical data, cancer stage, and laboratory data from postoperative days 1 to 3 were collected. Complications and readmission risk models were created using multivariable logistic regression and single-layer neural networks. SETTING National Cancer Institute-Designated Comprehensive Cancer Center. PATIENTS Adult patients with colorectal cancer. MAIN OUTCOME MEASURES The accuracy of predicting postoperative major complications, readmissions, and anastomotic leaks using the area under the receiver operating characteristic curve. RESULTS Neural networks had larger areas under the curve for predicting major complications compared to regression models (neural network 0.811; regression model 0.724, p < 0.001). Neural networks also showed an advantage in predicting anastomotic leak ( p = 0.036) and readmission using postoperative day 1 to 2 values ( p = 0.014). LIMITATIONS Single-center, retrospective design limited to cancer operations. CONCLUSIONS In this study, we generated a set of models for the early prediction of complications after colorectal surgery. The neural network models provided greater discrimination than the models based on traditional logistic regression. These models may allow for early detection of postoperative complications as early as postoperative day 2. See the Video Abstract . PREDICCIN POST OPERATORIA TEMPRANA DE COMPLICACIONES Y REINGRESO DESPUS DE LA CIRUGA DE CNCER COLORRECTAL MEDIANTE UNA RED NEURONAL ARTIFICIAL ANTECEDENTES:Los predictores tempranos de complicaciones postoperatorias pueden estratificar el riesgo de los pacientes sometidos a cirugía de cáncer colorrectal. Sin embargo, los modelos de regresión convencionales tienen un poder limitado para identificar relaciones no lineales complejas entre un gran conjunto de variables. Desarrollamos modelos de redes neuronales artificiales para optimizar la predicción de complicaciones postoperatorias importantes y riesgo de reingreso en pacientes sometidos a cirugía de cáncer colorrectal.OBJETIVO:El objetivo de este estudio fue desarrollar un modelo de red neuronal artificial para predecir complicaciones postoperatorias utilizando valores de laboratorio postoperatorios y comparar la precisión de estos modelos con los métodos de regresión estándar.DISEÑO:Este estudio retrospectivo incluyó a pacientes que se sometieron a resección electiva de cáncer colorrectal entre el 1 de enero de 2016 y el 31 de julio de 2021. Se recopilaron datos clínicos, estadio del cáncer y datos de laboratorio del día 1 al 3 posoperatorio. Se crearon modelos de complicaciones y riesgo de reingreso mediante regresión logística multivariable y redes neuronales de una sola capa.AJUSTE:Instituto Nacional del Cáncer designado Centro Oncológico Integral.PACIENTES:Pacientes adultos con cáncer colorrectal.PRINCIPALES MEDIDAS DE RESULTADO:Precisión de la predicción de complicaciones mayores postoperatorias, reingreso y fuga anastomótica utilizando el área bajo la curva característica operativa del receptor.RESULTADOS:Las redes neuronales tuvieron áreas bajo la curva más grandes para predecir complicaciones importantes en comparación con los modelos de regresión (red neuronal 0,811; modelo de regresión 0,724, p < 0,001). Las redes neuronales también mostraron una ventaja en la predicción de la fuga anastomótica ( p = 0,036) y el reingreso utilizando los valores del día 1-2 postoperatorio ( p = 0,014).LIMITACIONES:Diseño retrospectivo de un solo centro limitado a operaciones de cáncer.CONCLUSIONES:En este estudio, generamos un conjunto de modelos para la predicción temprana de complicaciones después de la cirugía colorrectal. Los modelos de redes neuronales proporcionaron una mayor discriminación que los modelos basados en regresión logística tradicional. Estos modelos pueden permitir la detección temprana de complicaciones posoperatorias tan pronto como el segundo día posoperatorio. (Traducción-Dr. Mauricio Santamaria ).
Collapse
Affiliation(s)
- Annamaria Agnes
- Department of General Surgery, Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Rome, Italy
| | - Sa T Nguyen
- Department of Colon and Rectal Surgery, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Tsuyoshi Konishi
- Department of Colon and Rectal Surgery, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Oliver Peacock
- Department of Colon and Rectal Surgery, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Brian K Bednarski
- Department of Colon and Rectal Surgery, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Y Nancy You
- Department of Colon and Rectal Surgery, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Craig A Messick
- Department of Colon and Rectal Surgery, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Matthew M Tillman
- Department of Colon and Rectal Surgery, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - John M Skibber
- Department of Colon and Rectal Surgery, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - George J Chang
- Department of Colon and Rectal Surgery, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Abhineet Uppal
- Department of Colon and Rectal Surgery, University of Texas MD Anderson Cancer Center, Houston, Texas
| |
Collapse
|
9
|
Shukla A, Chaudhary R, Nayyar N. Role of artificial intelligence in gastrointestinal surgery. Artif Intell Cancer 2024; 5. [DOI: 10.35713/aic.v5.i2.97317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/28/2024] [Revised: 07/11/2024] [Accepted: 07/17/2024] [Indexed: 09/05/2024] Open
Abstract
Artificial intelligence is rapidly evolving and its application is increasing day-by-day in the medical field. The application of artificial intelligence is also valuable in gastrointestinal diseases, by calculating various scoring systems, evaluating radiological images, preoperative and intraoperative assistance, processing pathological slides, prognosticating, and in treatment responses. This field has a promising future and can have an impact on many management algorithms. In this minireview, we aimed to determine the basics of artificial intelligence, the role that artificial intelligence may play in gastrointestinal surgeries and malignancies, and the limitations thereof.
Collapse
Affiliation(s)
- Ankit Shukla
- Department of Surgery, Dr Rajendra Prasad Government Medical College, Kangra 176001, Himachal Pradesh, India
| | - Rajesh Chaudhary
- Department of Renal Transplantation, Dr Rajendra Prasad Government Medical College, Kangra 176001, India
| | - Nishant Nayyar
- Department of Radiology, Dr Rajendra Prasad Government Medical College, Kangra 176001, Himachal Pradesh, India
| |
Collapse
|
10
|
Basile FV, Oliveira TS. Using Machine Learning to Select Breast Implant Volume. Plast Reconstr Surg 2024; 154:470e-477e. [PMID: 37843252 DOI: 10.1097/prs.0000000000011146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2023]
Abstract
BACKGROUND In breast augmentation surgery, selection of the appropriate breast implant size is a crucial step that can greatly affect patient satisfaction and the outcome of the procedure. However, this decision is often based on the subjective judgment of the surgeon and the patient, which can lead to suboptimal results. The authors aimed to develop a machine-learning approach that can accurately predict the size of breast implants selected for breast augmentation surgery. METHODS The authors collected data on patient demographic characteristics, medical history, and surgeon preferences from a sample of 1000 consecutive patients who underwent breast augmentation. This information was used to train and test a supervised machine-learning model to predict the size of breast implant needed. RESULTS The study demonstrated the effectiveness of the algorithm in predicting breast implant size, achieving a Pearson correlation coefficient of 0.9335 ( P < 0.001). The model generated accurate predictions in 86% of instances, with a mean absolute error of 27.10 mL. Its effectiveness was confirmed in the reoperation group, in which 36 of 57 patients (63%) would have received a more suitable implant size if the model's suggestion had been followed, potentially avoiding reoperation. CONCLUSIONS The findings show that machine learning can accurately predict the needed size of breast implants in augmentation surgery. By integrating the artificial intelligence model into a decision support system for breast augmentation surgery, essential guidance can be provided to surgeons and patients. This approach not only streamlines the implant selection process but also facilitates enhanced communication and decision-making, ultimately leading to more reliable outcomes and improved patient satisfaction.
Collapse
|
11
|
Bektaş M, Chia CM, Burchell GL, Daams F, Bonjer HJ, van der Peet DL. Artificial intelligence-aided ultrasound imaging in hepatopancreatobiliary surgery: where are we now? Surg Endosc 2024; 38:4869-4879. [PMID: 39160306 PMCID: PMC11362182 DOI: 10.1007/s00464-024-11130-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2024] [Accepted: 07/28/2024] [Indexed: 08/21/2024]
Abstract
BACKGROUND Artificial intelligence (AI) models have been applied in various medical imaging modalities and surgical disciplines, however the current status and progress of ultrasound-based AI models within hepatopancreatobiliary surgery have not been evaluated in literature. Therefore, this review aimed to provide an overview of ultrasound-based AI models used for hepatopancreatobiliary surgery, evaluating current advancements, validation, and predictive accuracies. METHOD Databases PubMed, EMBASE, Cochrane, and Web of Science were searched for studies using AI models on ultrasound for patients undergoing hepatopancreatobiliary surgery. To be eligible for inclusion, studies needed to apply AI methods on ultrasound imaging for patients undergoing hepatopancreatobiliary surgery. The Probast risk of bias tool was used to evaluate the methodological quality of AI methods. RESULTS AI models have been primarily used within hepatopancreatobiliary surgery, to predict tumor recurrence, differentiate between tumoral tissues, and identify lesions during ultrasound imaging. Most studies have combined radiomics with convolutional neural networks, with AUCs up to 0.98. CONCLUSION Ultrasound-based AI models have demonstrated promising accuracies in predicting early tumoral recurrence and even differentiating between tumoral tissue types during and after hepatopancreatobiliary surgery. However, prospective studies are required to evaluate if these results will remain consistent and externally valid.
Collapse
Affiliation(s)
- Mustafa Bektaş
- Amsterdam UMC Location Vrije Universiteit Amsterdam, Surgery, De Boelelaan 1117, Amsterdam, The Netherlands.
| | - Catherine M Chia
- Department of Computer Science, Vrije Universiteit Amsterdam, De Boelelaan 1105, Amsterdam, The Netherlands
| | - George L Burchell
- Amsterdam UMC Location Vrije Universiteit Amsterdam, Medical Library, De Boelelaan 1117, Amsterdam, The Netherlands
| | - Freek Daams
- Amsterdam UMC Location Vrije Universiteit Amsterdam, Surgery, De Boelelaan 1117, Amsterdam, The Netherlands
| | - H Jaap Bonjer
- Amsterdam UMC Location Vrije Universiteit Amsterdam, Surgery, De Boelelaan 1117, Amsterdam, The Netherlands
| | - Donald L van der Peet
- Amsterdam UMC Location Vrije Universiteit Amsterdam, Surgery, De Boelelaan 1117, Amsterdam, The Netherlands
| |
Collapse
|
12
|
Bektaş M, Pereira JK, Daams F, van der Peet DL. ChatGPT in surgery: a revolutionary innovation? Surg Today 2024; 54:964-971. [PMID: 38421439 PMCID: PMC11266448 DOI: 10.1007/s00595-024-02800-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 12/13/2023] [Indexed: 03/02/2024]
Abstract
ChatGPT has brought about a new era of digital health, as this model has become prominent and been rapidly developing since its release. ChatGPT may be able to facilitate improvements in surgery as well; however, the influence of ChatGPT on surgery is largely unknown at present. Therefore, the present study reports on the current applications of ChatGPT in the field of surgery, evaluating its workflow, practical implementations, limitations, and future perspectives. A literature search was performed using the PubMed and Embase databases. The initial search was performed from its inception until July 2023. This study revealed that ChatGPT has promising capabilities in areas of surgical research, education, training, and practice. In daily practice, surgeons and surgical residents can be aided in performing logistics and administrative tasks, and patients can be more efficiently informed about the details of their condition. However, priority should be given to establishing proper policies and protocols to ensure the safe and reliable use of this model.
Collapse
Affiliation(s)
- Mustafa Bektaş
- Amsterdam UMC Location Vrije Universiteit Amsterdam, Surgery, De Boelelaan 1117, Amsterdam, The Netherlands.
| | - Jaime Ken Pereira
- Department of Computer Science, Vrije Universiteit Amsterdam, De Boelelaan 1105, Amsterdam, The Netherlands
| | - Freek Daams
- Amsterdam UMC Location Vrije Universiteit Amsterdam, Surgery, De Boelelaan 1117, Amsterdam, The Netherlands
| | - Donald L van der Peet
- Amsterdam UMC Location Vrije Universiteit Amsterdam, Surgery, De Boelelaan 1117, Amsterdam, The Netherlands
| |
Collapse
|
13
|
Abdelhack M, Tripathi S, Chen Y, Avidan MS, King CR. Social vulnerability and surgery outcomes: a cross-sectional analysis. BMC Public Health 2024; 24:1907. [PMID: 39014400 PMCID: PMC11253435 DOI: 10.1186/s12889-024-19418-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2023] [Accepted: 07/09/2024] [Indexed: 07/18/2024] Open
Abstract
BACKGROUND Post-operative complications present a challenge to the healthcare system due to the high unpredictability of their incidence. Socioeconomic conditions have been established as social determinants of health. However, their contribution relating to postoperative complications is still unclear as it can be heterogeneous based on community, type of surgical services, and sex and gender. Uncovering these relations can enable improved public health policy to reduce such complications. METHODS In this study, we conducted a large population cross-sectional analysis of social vulnerability and the odds of various post-surgical complications. We collected electronic health records data from over 50,000 surgeries that happened between 2012 and 2018 at a quaternary health center in St. Louis, Missouri, United States and the corresponding zip code of the patients. We built statistical logistic regression models of postsurgical complications with the social vulnerability index of the tract consisting of the zip codes of the patient as the independent variable along with sex and race interaction. RESULTS Our sample from the St. Louis area exhibited high variance in social vulnerability with notable rapid increase in vulnerability from the south west to the north of the Mississippi river indicating high levels of inequality. Our sample had more females than males, and females had slightly higher social vulnerability index. Postoperative complication incidence ranged from 0.75% to 41% with lower incidence rate among females. We found that social vulnerability was associated with abnormal heart rhythm with socioeconomic status and housing status being the main association factors. We also found associations of the interaction of social vulnerability and female sex with an increase in odds of heart attack and surgical wound infection. Those associations disappeared when controlling for general health and comorbidities. CONCLUSIONS Our results indicate that social vulnerability measures such as socioeconomic status and housing conditions could affect postsurgical outcomes through preoperative health. This suggests that the domains of preventive medicine and public health should place social vulnerability as a priority to achieve better health outcomes of surgical interventions.
Collapse
Affiliation(s)
- Mohamed Abdelhack
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO, USA.
- Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health, Toronto, ON, Canada.
| | - Sandhya Tripathi
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO, USA.
| | - Yixin Chen
- Department of Computer Science, Washington University in St. Louis, St. Louis, MO, USA
| | - Michael S Avidan
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO, USA
| | - Christopher R King
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO, USA.
| |
Collapse
|
14
|
Brydges G, Chang GJ, Gan TJ, Konishi T, Gottumukkala V, Uppal A. Testing Machine Learning Models to Predict Postoperative Ileus after Colorectal Surgery. Curr Oncol 2024; 31:3563-3578. [PMID: 38920745 PMCID: PMC11202731 DOI: 10.3390/curroncol31060262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2024] [Revised: 05/08/2024] [Accepted: 05/15/2024] [Indexed: 06/27/2024] Open
Abstract
Background: Postoperative ileus (POI) is a common complication after colorectal surgery, leading to increased hospital stay and costs. This study aimed to explore patient comorbidities that contribute to the development of POI in the colorectal surgical population and compare machine learning (ML) model accuracy to existing risk instruments. Study Design: In a retrospective study, data were collected on 316 adult patients who underwent colorectal surgery from January 2020 to December 2021. The study excluded patients undergoing multi-visceral resections, re-operations, or combined primary and metastatic resections. Patients lacking follow-up within 90 days after surgery were also excluded. Eight different ML models were trained and cross-validated using 29 patient comorbidities and four comorbidity risk indices (ASA Status, NSQIP, CCI, and ECI). Results: The study found that 6.33% of patients experienced POI. Age, BMI, gender, kidney disease, anemia, arrhythmia, rheumatoid arthritis, and NSQIP score were identified as significant predictors of POI. The ML models with the greatest accuracy were AdaBoost tuned with grid search (94.2%) and XG Boost tuned with grid search (85.2%). Conclusions: This study suggests that ML models can predict the risk of POI with high accuracy and may offer a new frontier in early detection and intervention for postoperative outcome optimization. ML models can greatly improve the prediction and prevention of POI in colorectal surgery patients, which can lead to improved patient outcomes and reduced healthcare costs. Further research is required to validate and assess the replicability of these results.
Collapse
Affiliation(s)
- Garry Brydges
- Division of Anesthesiology, Critical Care & Pain Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (G.B.); (T.J.G.)
| | - George J. Chang
- Department of Colon & Rectal Surgery, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (G.J.C.); (T.K.); (A.U.)
| | - Tong J. Gan
- Division of Anesthesiology, Critical Care & Pain Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (G.B.); (T.J.G.)
| | - Tsuyoshi Konishi
- Department of Colon & Rectal Surgery, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (G.J.C.); (T.K.); (A.U.)
| | - Vijaya Gottumukkala
- Department of Anesthesiology & Perioperative Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Abhineet Uppal
- Department of Colon & Rectal Surgery, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; (G.J.C.); (T.K.); (A.U.)
| |
Collapse
|
15
|
Brydges G, Uppal A, Gottumukkala V. Application of Machine Learning in Predicting Perioperative Outcomes in Patients with Cancer: A Narrative Review for Clinicians. Curr Oncol 2024; 31:2727-2747. [PMID: 38785488 PMCID: PMC11120613 DOI: 10.3390/curroncol31050207] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2024] [Revised: 05/07/2024] [Accepted: 05/10/2024] [Indexed: 05/25/2024] Open
Abstract
This narrative review explores the utilization of machine learning (ML) and artificial intelligence (AI) models to enhance perioperative cancer care. ML and AI models offer significant potential to improve perioperative cancer care by predicting outcomes and supporting clinical decision-making. Tailored for perioperative professionals including anesthesiologists, surgeons, critical care physicians, nurse anesthetists, and perioperative nurses, this review provides a comprehensive framework for the integration of ML and AI models to enhance patient care delivery throughout the perioperative continuum.
Collapse
Affiliation(s)
- Garry Brydges
- Division of Anesthesiology, Critical Care & Pain Medicine, The University of Texas at MD Anderson Cancer Center, Houston, TX 77030, USA;
| | - Abhineet Uppal
- Department of Colon & Rectal Surgery, The University of Texas at MD Anderson Cancer Center, Houston, TX 77030, USA;
| | - Vijaya Gottumukkala
- Department of Anesthesiology & Perioperative Medicine, The University of Texas at MD Anderson Cancer Center, 1400-Unit 409, Holcombe Blvd, Houston, TX 77030, USA
| |
Collapse
|
16
|
Dixon D, Sattar H, Moros N, Kesireddy SR, Ahsan H, Lakkimsetti M, Fatima M, Doshi D, Sadhu K, Junaid Hassan M. Unveiling the Influence of AI Predictive Analytics on Patient Outcomes: A Comprehensive Narrative Review. Cureus 2024; 16:e59954. [PMID: 38854327 PMCID: PMC11161909 DOI: 10.7759/cureus.59954] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/08/2024] [Indexed: 06/11/2024] Open
Abstract
This comprehensive literature review explores the transformative impact of artificial intelligence (AI) predictive analytics on healthcare, particularly in improving patient outcomes regarding disease progression, treatment response, and recovery rates. AI, encompassing capabilities such as learning, problem-solving, and decision-making, is leveraged to predict disease progression, optimize treatment plans, and enhance recovery rates through the analysis of vast datasets, including electronic health records (EHRs), imaging, and genetic data. The utilization of machine learning (ML) and deep learning (DL) techniques in predictive analytics enables personalized medicine by facilitating the early detection of conditions, precision in drug discovery, and the tailoring of treatment to individual patient profiles. Ethical considerations, including data privacy, bias, and accountability, emerge as vital in the responsible implementation of AI in healthcare. The findings underscore the potential of AI predictive analytics in revolutionizing clinical decision-making and healthcare delivery, emphasizing the necessity of ethical guidelines and continuous model validation to ensure its safe and effective use in augmenting human judgment in medical practice.
Collapse
Affiliation(s)
- Diny Dixon
- Medicine, Jubilee Mission Medical College and Research Institute, Thrissur, IND
| | - Hina Sattar
- Medicine, Dow University of Health Sciences, Karachi, PAK
| | - Natalia Moros
- Medicine, Pontifical Javeriana University Medical School, Bogotá, COL
| | | | - Huma Ahsan
- Medicine, Jinnah Postgraduate Medical Centre, Karachi, PAK
| | | | - Madiha Fatima
- Medicine, Fatima Jinnah Medical University, Lahore, PAK
| | - Dhruvi Doshi
- Medicine, Gujarat Cancer Society Medical College, Hospital & Research Centre, Ahmedabad, IND
| | | | | |
Collapse
|
17
|
Nwaiwu CA, Rivera Perla KM, Abel LB, Sears IJ, Barton AT, Peterson RC, Liu YZ, Khatri IS, Sarkar IN, Shah N. Predicting Colonic Neoplasia Surgical Complications: A Machine Learning Approach. Dis Colon Rectum 2024; 67:700-713. [PMID: 38319746 DOI: 10.1097/dcr.0000000000003166] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/08/2024]
Abstract
BACKGROUND A range of statistical approaches have been used to help predict outcomes associated with colectomy. The multifactorial nature of complications suggests that machine learning algorithms may be more accurate in determining postoperative outcomes by detecting nonlinear associations, which are not readily measured by traditional statistics. OBJECTIVE The aim of this study was to investigate the utility of machine learning algorithms to predict complications in patients undergoing colectomy for colonic neoplasia. DESIGN Retrospective analysis using decision tree, random forest, and artificial neural network classifiers to predict postoperative outcomes. SETTINGS National Inpatient Sample database (2003-2017). PATIENTS Adult patients who underwent elective colectomy with anastomosis for neoplasia. MAIN OUTCOME MEASURES Performance was quantified using sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve to predict the incidence of anastomotic leak, prolonged length of stay, and inpatient mortality. RESULTS A total of 14,935 patients (4731 laparoscopic, 10,204 open) were included. They had an average age of 67 ± 12.2 years, and 53% of patients were women. The 3 machine learning models successfully identified patients who developed the measured complications. Although differences between model performances were largely insignificant, the neural network scored highest for most outcomes: predicting anastomotic leak, area under the receiver operating characteristic curve 0.88/0.93 (open/laparoscopic, 95% CI, 0.73-0.92/0.80-0.96); prolonged length of stay, area under the receiver operating characteristic curve 0.84/0.88 (open/laparoscopic, 95% CI, 0.82-0.85/0.85-0.91); and inpatient mortality, area under the receiver operating characteristic curve 0.90/0.92 (open/laparoscopic, 95% CI, 0.85-0.96/0.86-0.98). LIMITATIONS The patients from the National Inpatient Sample database may not be an accurate sample of the population of all patients undergoing colectomy for colonic neoplasia and does not account for specific institutional and patient factors. CONCLUSIONS Machine learning predicted postoperative complications in patients with colonic neoplasia undergoing colectomy with good performance. Although validation using external data and optimization of data quality will be required, these machine learning tools show great promise in assisting surgeons with risk-stratification of perioperative care to improve postoperative outcomes. See Video Abstract . PREDICCIN DE LAS COMPLICACIONES QUIRRGICAS DE LA NEOPLASIA DE COLON UN ENFOQUE DE MODELO DE APRENDIZAJE AUTOMTICO ANTECEDENTES:Se han utilizado una variedad de enfoques estadísticos para ayudar a predecir los resultados asociados con la colectomía. La naturaleza multifactorial de las complicaciones sugiere que los algoritmos de aprendizaje automático pueden ser más precisos en determinar los resultados posoperatorios al detectar asociaciones no lineales, que generalmente no se miden en las estadísticas tradicionales.OBJETIVO:El objetivo de este estudio fue investigar la utilidad de los algoritmos de aprendizaje automático para predecir complicaciones en pacientes sometidos a colectomía por neoplasia de colon.DISEÑO:Análisis retrospectivo utilizando clasificadores de árboles de decisión, bosques aleatorios y redes neuronales artificiales para predecir los resultados posoperatorios.AJUSTE:Base de datos de la Muestra Nacional de Pacientes Hospitalizados (2003-2017).PACIENTES:Pacientes adultos sometidos a colectomía electiva con anastomosis por neoplasia.INTERVENCIONES:N/A.PRINCIPALES MEDIDAS DE RESULTADO:El rendimiento se cuantificó utilizando la sensibilidad, especificidad, precisión y la característica operativa del receptor del área bajo la curva para predecir la incidencia de fuga anastomótica, duración prolongada de la estancia hospitalaria y mortalidad de los pacientes hospitalizados.RESULTADOS:Se incluyeron un total de 14.935 pacientes (4.731 laparoscópicos, 10.204 abiertos). Presentaron una edad promedio de 67 ± 12,2 años y el 53% eran mujeres. Los tres modelos de aprendizaje automático identificaron con éxito a los pacientes que desarrollaron las complicaciones medidas. Aunque las diferencias entre el rendimiento del modelo fueron en gran medida insignificantes, la red neuronal obtuvo la puntuación más alta para la mayoría de los resultados: predicción de fuga anastomótica, característica operativa del receptor del área bajo la curva 0,88/0,93 (abierta/laparoscópica, IC del 95%: 0,73-0,92/0,80-0,96); duración prolongada de la estancia hospitalaria, característica operativa del receptor del área bajo la curva 0,84/0,88 (abierta/laparoscópica, IC del 95%: 0,82-0,85/0,85-0,91); y mortalidad de pacientes hospitalizados, característica operativa del receptor del área bajo la curva 0,90/0,92 (abierto/laparoscópico, IC del 95%: 0,85-0,96/0,86-0,98).LIMITACIONES:Los pacientes de la base de datos de la Muestra Nacional de Pacientes Hospitalizados pueden no ser una muestra precisa de la población de todos los pacientes sometidos a colectomía por neoplasia de colon y no tienen en cuenta factores institucionales y específicos del paciente.CONCLUSIONES:El aprendizaje automático predijo con buen rendimiento las complicaciones postoperatorias en pacientes con neoplasia de colon sometidos a colectomía. Aunque será necesaria la validación mediante datos externos y la optimización de la calidad de los datos, estas herramientas de aprendizaje automático son muy prometedoras para ayudar a los cirujanos con la estratificación de riesgos de la atención perioperatoria para mejorar los resultados posoperatorios. (Traducción-Dr. Fidel Ruiz Healy ).
Collapse
Affiliation(s)
- Chibueze A Nwaiwu
- Department of Surgery, Warren Alpert Medical School of Brown University, Rhode Island Hospital, Providence, Rhode Island
| | - Krissia M Rivera Perla
- Department of Surgery, Warren Alpert Medical School of Brown University, Rhode Island Hospital, Providence, Rhode Island
- Harvard T.H. Chan School of Public Health, Boston, Massachusetts
| | - Logan B Abel
- Department of Surgery, Warren Alpert Medical School of Brown University, Rhode Island Hospital, Providence, Rhode Island
| | - Isaac J Sears
- Department of Surgery, Warren Alpert Medical School of Brown University, Rhode Island Hospital, Providence, Rhode Island
| | - Andrew T Barton
- Department of Surgery, Warren Alpert Medical School of Brown University, Rhode Island Hospital, Providence, Rhode Island
| | | | - Yao Z Liu
- Department of Surgery, Warren Alpert Medical School of Brown University, Rhode Island Hospital, Providence, Rhode Island
| | - Ishaani S Khatri
- Department of Surgery, Warren Alpert Medical School of Brown University, Rhode Island Hospital, Providence, Rhode Island
| | - Indra N Sarkar
- Center for Biomedical Informatics, Brown University, Providence, Rhode Island
- Rhode Island Quality Institute, Providence, Rhode Island
| | - Nishit Shah
- Department of Surgery, Warren Alpert Medical School of Brown University, Rhode Island Hospital, Providence, Rhode Island
| |
Collapse
|
18
|
Hernandez MC, Chen C, Nguyen A, Choong K, Carlin C, Nelson RA, Rossi LA, Seth N, McNeese K, Yuh B, Eftekhari Z, Lai LL. Explainable Machine Learning Model to Preoperatively Predict Postoperative Complications in Inpatients With Cancer Undergoing Major Operations. JCO Clin Cancer Inform 2024; 8:e2300247. [PMID: 38648576 PMCID: PMC11161247 DOI: 10.1200/cci.23.00247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Revised: 01/24/2024] [Accepted: 03/06/2024] [Indexed: 04/25/2024] Open
Abstract
PURPOSE Preoperative prediction of postoperative complications (PCs) in inpatients with cancer is challenging. We developed an explainable machine learning (ML) model to predict PCs in a heterogenous population of inpatients with cancer undergoing same-hospitalization major operations. METHODS Consecutive inpatients who underwent same-hospitalization operations from December 2017 to June 2021 at a single institution were retrospectively reviewed. The ML model was developed and tested using electronic health record (EHR) data to predict 30-day PCs for patients with Clavien-Dindo grade 3 or higher (CD 3+) per the CD classification system. Model performance was assessed using area under the receiver operating characteristic curve (AUROC), area under the precision recall curve (AUPRC), and calibration plots. Model explanation was performed using the Shapley additive explanations (SHAP) method at cohort and individual operation levels. RESULTS A total of 988 operations in 827 inpatients were included. The ML model was trained using 788 operations and tested using a holdout set of 200 operations. The CD 3+ complication rates were 28.6% and 27.5% in the training and holdout test sets, respectively. Training and holdout test sets' model performance in predicting CD 3+ complications yielded an AUROC of 0.77 and 0.73 and an AUPRC of 0.56 and 0.52, respectively. Calibration plots demonstrated good reliability. The SHAP method identified features and the contributions of the features to the risk of PCs. CONCLUSION We trained and tested an explainable ML model to predict the risk of developing PCs in patients with cancer. Using patient-specific EHR data, the ML model accurately discriminated the risk of developing CD 3+ complications and displayed top features at the individual operation and cohort level.
Collapse
Affiliation(s)
| | - Chen Chen
- Department of Applied AI and Data Science, City of Hope National Medical Center, Duarte, CA
| | - Andrew Nguyen
- Department of Surgery, City of Hope National Medical Center, Duarte, CA
| | - Kevin Choong
- Department of Surgery, Division of Oncology, Primas Health, University of South Carolina Medical School, Greeneville, SC
| | - Cameron Carlin
- Department of Applied AI and Data Science, City of Hope National Medical Center, Duarte, CA
| | - Rebecca A. Nelson
- Department of Computational and Quantitative Medicine, Division of Biostatistics, City of Hope National Medical Center, Duarte, CA
| | - Lorenzo A. Rossi
- Department of Applied AI and Data Science, City of Hope National Medical Center, Duarte, CA
| | - Naini Seth
- Department of Clinical Informatics, City of Hope National Medical Center, Duarte, CA
| | - Kathy McNeese
- Department of Surgery, University of New Mexico, Albuquerque, NM
| | - Bertram Yuh
- Department of Surgery, University of New Mexico, Albuquerque, NM
| | - Zahra Eftekhari
- Department of Applied AI and Data Science, City of Hope National Medical Center, Duarte, CA
| | - Lily L. Lai
- Department of Surgery, University of New Mexico, Albuquerque, NM
| |
Collapse
|
19
|
Chen KA, Goffredo P, Butler LR, Joisa CU, Guillem JG, Gomez SM, Kapadia MR. Prediction of Pathologic Complete Response for Rectal Cancer Based on Pretreatment Factors Using Machine Learning. Dis Colon Rectum 2024; 67:387-397. [PMID: 37994445 PMCID: PMC11186794 DOI: 10.1097/dcr.0000000000003038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/24/2023]
Abstract
BACKGROUND Pathologic complete response after neoadjuvant therapy is an important prognostic indicator for locally advanced rectal cancer and may give insights into which patients might be treated nonoperatively in the future. Existing models for predicting pathologic complete response in the pretreatment setting are limited by small data sets and low accuracy. OBJECTIVE We sought to use machine learning to develop a more generalizable predictive model for pathologic complete response for locally advanced rectal cancer. DESIGN Patients with locally advanced rectal cancer who underwent neoadjuvant therapy followed by surgical resection were identified in the National Cancer Database from years 2010 to 2019 and were split into training, validation, and test sets. Machine learning techniques included random forest, gradient boosting, and artificial neural network. A logistic regression model was also created. Model performance was assessed using an area under the receiver operating characteristic curve. SETTINGS This study used a national, multicenter data set. PATIENTS Patients with locally advanced rectal cancer who underwent neoadjuvant therapy and proctectomy. MAIN OUTCOME MEASURES Pathologic complete response defined as T0/xN0/x. RESULTS The data set included 53,684 patients. Pathologic complete response was experienced by 22.9% of patients. Gradient boosting showed the best performance with an area under the receiver operating characteristic curve of 0.777 (95% CI, 0.773-0.781), compared with 0.684 (95% CI, 0.68-0.688) for logistic regression. The strongest predictors of pathologic complete response were no lymphovascular invasion, no perineural invasion, lower CEA, smaller size of tumor, and microsatellite stability. A concise model including the top 5 variables showed preserved performance. LIMITATIONS The models were not externally validated. CONCLUSIONS Machine learning techniques can be used to accurately predict pathologic complete response for locally advanced rectal cancer in the pretreatment setting. After fine-tuning a data set including patients treated nonoperatively, these models could help clinicians identify the appropriate candidates for a watch-and-wait strategy. See Video Abstract . EL CNCER DE RECTO BASADA EN FACTORES PREVIOS AL TRATAMIENTO MEDIANTE EL APRENDIZAJE AUTOMTICO ANTECEDENTES:La respuesta patológica completa después de la terapia neoadyuvante es un indicador pronóstico importante para el cáncer de recto localmente avanzado y puede dar información sobre qué pacientes podrían ser tratados de forma no quirúrgica en el futuro. Los modelos existentes para predecir la respuesta patológica completa en el entorno previo al tratamiento están limitados por conjuntos de datos pequeños y baja precisión.OBJETIVO:Intentamos utilizar el aprendizaje automático para desarrollar un modelo predictivo más generalizable para la respuesta patológica completa para el cáncer de recto localmente avanzado.DISEÑO:Los pacientes con cáncer de recto localmente avanzado que se sometieron a terapia neoadyuvante seguida de resección quirúrgica se identificaron en la Base de Datos Nacional del Cáncer de los años 2010 a 2019 y se dividieron en conjuntos de capacitación, validación y prueba. Las técnicas de aprendizaje automático incluyeron bosque aleatorio, aumento de gradiente y red neuronal artificial. También se creó un modelo de regresión logística. El rendimiento del modelo se evaluó utilizando el área bajo la curva característica operativa del receptor.ÁMBITO:Este estudio utilizó un conjunto de datos nacional multicéntrico.PACIENTES:Pacientes con cáncer de recto localmente avanzado sometidos a terapia neoadyuvante y proctectomía.PRINCIPALES MEDIDAS DE VALORACIÓN:Respuesta patológica completa definida como T0/xN0/x.RESULTADOS:El conjunto de datos incluyó 53.684 pacientes. El 22,9% de los pacientes experimentaron una respuesta patológica completa. El refuerzo de gradiente mostró el mejor rendimiento con un área bajo la curva característica operativa del receptor de 0,777 (IC del 95%: 0,773 - 0,781), en comparación con 0,684 (IC del 95%: 0,68 - 0,688) para la regresión logística. Los predictores más fuertes de respuesta patológica completa fueron la ausencia de invasión linfovascular, la ausencia de invasión perineural, un CEA más bajo, un tamaño más pequeño del tumor y la estabilidad de los microsatélites. Un modelo conciso que incluye las cinco variables principales mostró un rendimiento preservado.LIMITACIONES:Los modelos no fueron validados externamente.CONCLUSIONES:Las técnicas de aprendizaje automático se pueden utilizar para predecir con precisión la respuesta patológica completa para el cáncer de recto localmente avanzado en el entorno previo al tratamiento. Después de realizar ajustes en un conjunto de datos que incluye pacientes tratados de forma no quirúrgica, estos modelos podrían ayudar a los médicos a identificar a los candidatos adecuados para una estrategia de observar y esperar. (Traducción-Dr. Ingrid Melo ).
Collapse
Affiliation(s)
- Kevin A Chen
- Division of Gastrointestinal Surgery, Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill, NC, 4038 Burnett Womack Building, Chapel Hill, NC 27599
| | - Paolo Goffredo
- Division of Colorectal Surgery, Department of Surgery, University of Minnesota, Minneapolis, MN, 420 Delaware St SE, Minneapolis, MN 55455
| | - Logan R Butler
- Division of Gastrointestinal Surgery, Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill, NC, 4038 Burnett Womack Building, Chapel Hill, NC 27599
| | - Chinmaya U Joisa
- Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill, Chapel Hill, NC, 10202C Mary Ellen Jones Building, Chapel Hill, NC, 27599
| | - Jose G Guillem
- Division of Gastrointestinal Surgery, Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill, NC, 4038 Burnett Womack Building, Chapel Hill, NC 27599
| | - Shawn M Gomez
- Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill, Chapel Hill, NC, 10202C Mary Ellen Jones Building, Chapel Hill, NC, 27599
| | - Muneera R Kapadia
- Division of Gastrointestinal Surgery, Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill, NC, 4038 Burnett Womack Building, Chapel Hill, NC 27599
| |
Collapse
|
20
|
Koenen L, Arens P, Olze H, Dommerich S. Classifying and Predicting Surgical Complications After Laryngectomy: A Novel Approach to Diagnosing and Treating Patients. EAR, NOSE & THROAT JOURNAL 2024; 103:NP53-NP59. [PMID: 34328819 DOI: 10.1177/01455613211029749] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
OBJECTIVES The total laryngectomy is one of the most standardized major surgical procedures in otolaryngology. Several studies have proposed the Clavien-Dindo classification (CDC) as a solution to classifying postoperative complications into 5 grades from less severe to severe. Yet more data on classifying larger patient populations undergoing major otolaryngologic surgery according to the CDC are needed. Predicting postoperative complications in clinical practice is often subject to generalized clinical scoring systems with uncertain predictive abilities for otolaryngologic surgery. Machine learning offers methods to predict postoperative complications based on data obtained prior to surgery. METHODS We included all patients (N = 148) who underwent a total laryngectomy after diagnosis of squamous cell carcinoma at our institution. A univariate and multivariate logistic regression analysis of multiple complex risk factors was performed, and patients were grouped into severe postoperative complications (CDC ≥ 4) and less severe complications. Four different commonly used machine learning algorithms were trained on the dataset. The best model was selected to predict postoperative complications on the complete dataset. RESULTS Univariate analysis showed that the most significant predictors for postoperative complications were the Charlson Comorbidity Index (CCI) and whether reconstruction was performed intraoperatively. A multivariate analysis showed that the CCI and reconstruction remained significant. The commonly used AdaBoost algorithm achieved the highest area under the curve with 0.77 with high positive and negative predictive values in subsequent analysis. CONCLUSIONS This study shows that postoperative complications can be classified according to the CDC with the CCI being a useful screening tool to predict patients at risk for postoperative complications. We provide evidence that could help identify single patients at risk for complications and customize treatment accordingly which could finally lead to a custom approach for every patient. We also suggest that there is no increase in complications with patients of higher age.
Collapse
Affiliation(s)
- Lukas Koenen
- Department of Otolaryngology, Charité - Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Philipp Arens
- Department of Otolaryngology, Charité - Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Heidi Olze
- Department of Otolaryngology, Charité - Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Steffen Dommerich
- Department of Otolaryngology, Charité - Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| |
Collapse
|
21
|
Liu J, Cao B, Luo Y, Chen X, Han H, Li L, Zeng J. Risk factors of major bleeding detected by machine learning method in patients undergoing liver resection with controlled low central venous pressure technique. Postgrad Med J 2023; 99:1280-1286. [PMID: 37794600 DOI: 10.1093/postmj/qgad087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 08/18/2023] [Accepted: 09/01/2023] [Indexed: 10/06/2023]
Abstract
BACKGROUND Controlled low central venous pressure (CLCVP) technique has been extensively validated in clinical practices to decrease intraoperative bleeding during liver resection process; however, no studies to date have attempted to propose a scoring method to better understand what risk factors might still be responsible for bleeding when CLCVP technique was implemented. METHODS We aimed to use machine learning to develop a model for detecting the risk factors of major bleeding in patients who underwent liver resection using CLCVP technique. We reviewed the medical records of 1077 patients who underwent liver surgery between January 2017 and June 2020. We evaluated the XGBoost model and logistic regression model using stratified K-fold cross-validation (K = 5), and the area under the receiver operating characteristic curve, the recall rate, precision rate, and accuracy score were calculated and compared. The SHapley Additive exPlanations was employed to identify the most influencing factors and their contribution to the prediction. RESULTS The XGBoost classifier with an accuracy of 0.80 and precision of 0.89 outperformed the logistic regression model with an accuracy of 0.76 and precision of 0.79. According to the SHapley Additive exPlanations summary plot, the top six variables ranked from most to least important included intraoperative hematocrit, surgery duration, intraoperative lactate, preoperative hemoglobin, preoperative aspartate transaminase, and Pringle maneuver duration. CONCLUSIONS Anesthesiologists should be aware of the potential impact of increased Pringle maneuver duration and lactate levels on intraoperative major bleeding in patients undergoing liver resection with CLCVP technique. What is already known on this topic-Low central venous pressure technique has already been extensively validated in clinical practices, with no prediction model for major bleeding. What this study adds-The XGBoost classifier outperformed logistic regression model for the prediction of major bleeding during liver resection with low central venous pressure technique. How this study might affect research, practice, or policy-anesthesiologists should be aware of the potential impact of increased PM duration and lactate levels on intraoperative major bleeding in patients undergoing liver resection with CLCVP technique.
Collapse
Affiliation(s)
- Jing Liu
- Department of Anesthesiology, the Eighth Affiliated Hospital, Sun Yat-sen University, Shenzhen 518033, China
| | - Bingbing Cao
- Department of Anesthesiology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou 510000, China
| | - Yuelian Luo
- Department of Anesthesiology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou 510000, China
| | - Xianqing Chen
- Department of Hepatobiliary and Pancreatic Surgery, the Eighth Affiliated Hospital, Sun Yat-sen University, Shenzhen 518033, China
| | - Hong Han
- Department of Anesthesiology, the Eighth Affiliated Hospital, Sun Yat-sen University, Shenzhen 518033, China
| | - Li Li
- Department of Anesthesiology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou 510000, China
| | - Jianfeng Zeng
- Department of Anesthesiology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou 510000, China
| |
Collapse
|
22
|
Butler LR, Chen KA, Hsu J, Kapadia MR, Gomez SM, Farrell TM. Predicting readmission after bariatric surgery using machine learning. Surg Obes Relat Dis 2023; 19:1236-1244. [PMID: 37455158 PMCID: PMC12057647 DOI: 10.1016/j.soard.2023.05.025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Accepted: 05/27/2023] [Indexed: 07/18/2023]
Abstract
BACKGROUND While bariatric surgery is an effective method for achieving long-term weight loss, postoperative readmissions are associated with negative clinical outcomes and significant costs. OBJECTIVES We aimed to use machine learning (ML) algorithms to predict readmissions and compare results to logistic regression. SETTING Hospitals participating in the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program, United States. METHODS Patients who underwent sleeve gastrectomy (SG), Roux-en-Y gastric bypass (RYGB), and biliopancreatic diversion with duodenal switch between 2016 and 2020 were selected from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP) database. Patient variables reported by the MBSAQIP database were analyzed by ML algorithms random forest (RF), gradient boosting (XGB), and deep neural networks (NN), and the results of the predictive models were compared to logistic regression using area under the receiver operating characteristic curve (AUROC). RESULTS Our study included 863,348 patients, of which 39,068 (4.52%) were readmitted. AUROC scores were XGB .785 (95% CI .784-.786), RF .785 (95% CI .784-.785), and NN .754 (95% CI .753-.754), compared with .62 (95% CI .62-.621) for logistic regression (LR) (P < .001). The sensitivity and specificity for XGB, the best performing model, were 73.81% and 70%, compared with 52.94% and 70% for logistic regression. The most important variables were intervention or reoperation prior to discharge, unplanned ICU admission, initial procedure, and the intraoperative transfusion. CONCLUSIONS ML demonstrates significant advantages over logistic regression when predicting 30-day readmission following bariatric surgery. With external validation, models could identify the best candidates for early discharge or targeted postdischarge resources.
Collapse
Affiliation(s)
- Logan R Butler
- Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina.
| | - Kevin A Chen
- Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Justin Hsu
- Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Muneera R Kapadia
- Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Shawn M Gomez
- Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Timothy M Farrell
- Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| |
Collapse
|
23
|
Stam WT, Ingwersen EW, Ali M, Spijkerman JT, Kazemier G, Bruns ERJ, Daams F. Machine learning models in clinical practice for the prediction of postoperative complications after major abdominal surgery. Surg Today 2023; 53:1209-1215. [PMID: 36840764 PMCID: PMC10520164 DOI: 10.1007/s00595-023-02662-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Accepted: 02/07/2023] [Indexed: 02/26/2023]
Abstract
Complications after surgery have a major impact on short- and long-term outcomes, and decades of technological advancement have not yet led to the eradication of their risk. The accurate prediction of complications, recently enhanced by the development of machine learning algorithms, has the potential to completely reshape surgical patient management. In this paper, we reflect on multiple issues facing the implementation of machine learning, from the development to the actual implementation of machine learning models in daily clinical practice, providing suggestions on the use of machine learning models for predicting postoperative complications after major abdominal surgery.
Collapse
Affiliation(s)
- Wessel T Stam
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands
- AGEM Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam, The Netherlands
| | - Erik W Ingwersen
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands
- AGEM Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam, The Netherlands
| | - Mahsoem Ali
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands
| | - Jorik T Spijkerman
- Independent Consultant in Computational Intelligence, Amsterdam, The Netherlands
| | - Geert Kazemier
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands
| | - Emma R J Bruns
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands
| | - Freek Daams
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands.
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands.
| |
Collapse
|
24
|
Kim JH, Cheon BR, Kim MG, Hwang SM, Lim SY, Lee JJ, Kwon YS. Postoperative Nausea and Vomiting Prediction: Machine Learning Insights from a Comprehensive Analysis of Perioperative Data. Bioengineering (Basel) 2023; 10:1152. [PMID: 37892882 PMCID: PMC10604280 DOI: 10.3390/bioengineering10101152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 09/27/2023] [Accepted: 09/28/2023] [Indexed: 10/29/2023] Open
Abstract
Postoperative nausea and vomiting (PONV) are common complications after surgery. This study aimed to present the utilization of machine learning for predicting PONV and provide insights based on a large amount of data. This retrospective study included data on perioperative features of patients, such as patient characteristics and perioperative factors, from two hospitals. Logistic regression algorithms, random forest, light-gradient boosting machines, and multilayer perceptrons were used as machine learning algorithms to develop the models. The dataset of this study included 106,860 adult patients, with an overall incidence rate of 14.4% for PONV. The area under the receiver operating characteristic curve (AUROC) of the models was 0.60-0.67. In the prediction models that included only the known risk and mitigating factors of PONV, the AUROC of the models was 0.54-0.69. Some features were found to be associated with patient-controlled analgesia, with opioids being the most important feature in almost all models. In conclusion, machine learning provides valuable insights into PONV prediction, the selection of significant features for prediction, and feature engineering.
Collapse
Affiliation(s)
- Jong-Ho Kim
- Department of Anesthesiology and Pain Medicine, Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon-si 24253, Republic of Korea; (J.-H.K.); (B.-R.C.); (M.-G.K.); (S.-M.H.); (S.-Y.L.); (J.-J.L.)
- Institute of New Frontier Research Team, Hallym University College of Medicine, Chuncheon-si 24252, Republic of Korea
| | - Bo-Reum Cheon
- Department of Anesthesiology and Pain Medicine, Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon-si 24253, Republic of Korea; (J.-H.K.); (B.-R.C.); (M.-G.K.); (S.-M.H.); (S.-Y.L.); (J.-J.L.)
| | - Min-Guan Kim
- Department of Anesthesiology and Pain Medicine, Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon-si 24253, Republic of Korea; (J.-H.K.); (B.-R.C.); (M.-G.K.); (S.-M.H.); (S.-Y.L.); (J.-J.L.)
| | - Sung-Mi Hwang
- Department of Anesthesiology and Pain Medicine, Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon-si 24253, Republic of Korea; (J.-H.K.); (B.-R.C.); (M.-G.K.); (S.-M.H.); (S.-Y.L.); (J.-J.L.)
| | - So-Young Lim
- Department of Anesthesiology and Pain Medicine, Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon-si 24253, Republic of Korea; (J.-H.K.); (B.-R.C.); (M.-G.K.); (S.-M.H.); (S.-Y.L.); (J.-J.L.)
| | - Jae-Jun Lee
- Department of Anesthesiology and Pain Medicine, Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon-si 24253, Republic of Korea; (J.-H.K.); (B.-R.C.); (M.-G.K.); (S.-M.H.); (S.-Y.L.); (J.-J.L.)
- Institute of New Frontier Research Team, Hallym University College of Medicine, Chuncheon-si 24252, Republic of Korea
| | - Young-Suk Kwon
- Department of Anesthesiology and Pain Medicine, Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, Chuncheon-si 24253, Republic of Korea; (J.-H.K.); (B.-R.C.); (M.-G.K.); (S.-M.H.); (S.-Y.L.); (J.-J.L.)
- Institute of New Frontier Research Team, Hallym University College of Medicine, Chuncheon-si 24252, Republic of Korea
| |
Collapse
|
25
|
Abu-Khudir R, Hafsa N, Badr BE. Identifying Effective Biomarkers for Accurate Pancreatic Cancer Prognosis Using Statistical Machine Learning. Diagnostics (Basel) 2023; 13:3091. [PMID: 37835833 PMCID: PMC10572229 DOI: 10.3390/diagnostics13193091] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Revised: 09/08/2023] [Accepted: 09/26/2023] [Indexed: 10/15/2023] Open
Abstract
Pancreatic cancer (PC) has one of the lowest survival rates among all major types of cancer. Consequently, it is one of the leading causes of mortality worldwide. Serum biomarkers historically correlate well with the early prognosis of post-surgical complications of PC. However, attempts to identify an effective biomarker panel for the successful prognosis of PC were almost non-existent in the current literature. The current study investigated the roles of various serum biomarkers including carbohydrate antigen 19-9 (CA19-9), chemokine (C-X-C motif) ligand 8 (CXCL-8), procalcitonin (PCT), and other relevant clinical data for identifying PC progression, classified into sepsis, recurrence, and other post-surgical complications, among PC patients. The most relevant biochemical and clinical markers for PC prognosis were identified using a random-forest-powered feature elimination method. Using this informative biomarker panel, the selected machine-learning (ML) classification models demonstrated highly accurate results for classifying PC patients into three complication groups on independent test data. The superiority of the combined biomarker panel (Max AUC-ROC = 100%) was further established over using CA19-9 features exclusively (Max AUC-ROC = 75%) for the task of classifying PC progression. This novel study demonstrates the effectiveness of the combined biomarker panel in successfully diagnosing PC progression and other relevant complications among Egyptian PC survivors.
Collapse
Affiliation(s)
- Rasha Abu-Khudir
- Chemistry Department, College of Science, King Faisal University, P.O. Box 380, Hofuf 31982, Al-Ahsa, Saudi Arabia
- Chemistry Department, Biochemistry Branch, Faculty of Science, Tanta University, Tanta 31527, Egypt
| | - Noor Hafsa
- Computer Science Department, College of Computer Science and Information Technology, King Faisal University, P.O. Box 400, Hofuf 31982, Al-Ahsa, Saudi Arabia;
| | - Badr E. Badr
- Egyptian Ministry of Labor, Training and Research Department, Tanta 31512, Egypt;
- Botany Department, Microbiology Unit, Faculty of Science, Tanta University, Tanta 31527, Egypt
| |
Collapse
|
26
|
Ingwersen EW, Stam WT, Meijs BJV, Roor J, Besselink MG, Groot Koerkamp B, de Hingh IHJT, van Santvoort HC, Stommel MWJ, Daams F. Machine learning versus logistic regression for the prediction of complications after pancreatoduodenectomy. Surgery 2023; 174:435-440. [PMID: 37150712 DOI: 10.1016/j.surg.2023.03.012] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 03/02/2023] [Accepted: 03/20/2023] [Indexed: 05/09/2023]
Abstract
BACKGROUND Machine learning is increasingly advocated to develop prediction models for postoperative complications. It is, however, unclear if machine learning is superior to logistic regression when using structured clinical data. Postoperative pancreatic fistula and delayed gastric emptying are the two most common complications with the biggest impact on patient condition and length of hospital stay after pancreatoduodenectomy. This study aimed to compare the performance of machine learning and logistic regression in predicting pancreatic fistula and delayed gastric emptying after pancreatoduodenectomy. METHODS This retrospective observational study used nationwide data from 16 centers in the Dutch Pancreatic Cancer Audit between January 2014 and January 2021. The area under the curve of a machine learning and logistic regression model for clinically relevant postoperative pancreatic fistula and delayed gastric emptying were compared. RESULTS Overall, 799 (16.3%) patients developed a postoperative pancreatic fistula, and 943 developed (19.2%) delayed gastric emptying. For postoperative pancreatic fistula, the area under the curve of the machine learning model was 0.74, and the area under the curve of the logistic regression model was 0.73. For delayed gastric emptying, the area under the curve of the machine learning model and logistic regression was 0.59. CONCLUSION Machine learning did not outperform logistic regression modeling in predicting postoperative complications after pancreatoduodenectomy.
Collapse
Affiliation(s)
- Erik W Ingwersen
- Amsterdam UMC, location Vrije Universiteit Amsterdam, Department of Surgery, Amsterdam, the Netherlands; Cancer Center Amsterdam, the Netherlands; Amsterdam Gastroenterology Endocrinology and Metabolism, the Netherlands
| | - Wessel T Stam
- Amsterdam UMC, location Vrije Universiteit Amsterdam, Department of Surgery, Amsterdam, the Netherlands; Cancer Center Amsterdam, the Netherlands; Amsterdam Gastroenterology Endocrinology and Metabolism, the Netherlands
| | - Bono J V Meijs
- Amsterdam UMC, location Vrije Universiteit Amsterdam, Department of Surgery, Amsterdam, the Netherlands; Cancer Center Amsterdam, the Netherlands; Amsterdam Gastroenterology Endocrinology and Metabolism, the Netherlands
| | - Joran Roor
- SAS institute B.V., Huizen, the Netherlands
| | - Marc G Besselink
- Cancer Center Amsterdam, the Netherlands; Amsterdam Gastroenterology Endocrinology and Metabolism, the Netherlands; Department of Surgery, Amsterdam UMC, University of Amsterdam, the Netherlands
| | | | | | | | | | - Freek Daams
- Amsterdam UMC, location Vrije Universiteit Amsterdam, Department of Surgery, Amsterdam, the Netherlands; Cancer Center Amsterdam, the Netherlands.
| |
Collapse
|
27
|
Chen KA, Goffredo P, Hu D, Joisa CU, Guillem JG, Gomez SM, Kapadia MR. Estimating Risk of Locoregional Failure and Overall Survival in Anal Cancer Following Chemoradiation: A Machine Learning Approach. J Gastrointest Surg 2023; 27:1925-1935. [PMID: 37407899 PMCID: PMC10528925 DOI: 10.1007/s11605-023-05755-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 06/03/2023] [Indexed: 07/07/2023]
Abstract
BACKGROUND Optimal treatment of anal squamous cell carcinoma (ASCC) is definitive chemoradiation. Patients with persistent or recurrent disease require abdominoperineal resection (APR). Current models for predicting need for APR and overall survival are limited by low accuracy or small datasets. This study sought to use machine learning (ML) to develop more accurate models for locoregional failure and overall survival for ASCC. METHODS This study used the National Cancer Database from 2004-2018, divided into training, validation, and test sets. We included patients with stage I-III ASCC who underwent chemoradiation. Our primary outcomes were need for APR and 3-year overall survival. Random forest (RF), gradient boosting (XGB), and neural network (NN) ML-based models were developed and compared with logistic regression (LR). Accuracy was assessed using area under the receiver operating characteristic curve (AUROC). RESULTS APR was required in 5.3% (1,015/18,978) of patients. XGB performed best with AUROC of 0.813, compared with 0.691 for LR. Tumor size, lymphovascular invasion, and tumor grade showed the strongest influence on model predictions. Mortality was 23.6% (7,988/33,834). AUROC for XGB and LR were similar at 0.766 and 0.748, respectively. For this model, age, radiation dose, sex, and insurance status were the most influential variables. CONCLUSIONS We developed and internally validated machine learning-based models for predicting outcomes in ASCC and showed higher accuracy versus LR for locoregional failure, but not overall survival. After external validation, these models may assist clinicians with identifying patients with ASCC at high risk of treatment failure.
Collapse
Affiliation(s)
- Kevin A Chen
- Divison of Gastrointestinal Surgery, Department of Surgery, University of North Carolina at Chapel Hill, 100 Manning Drive, 4038 Burnett Womack Building, CB #7050, Chapel Hill, NC, 27599, USA
| | - Paolo Goffredo
- Division of Colon & Rectal Surgery, Department of Surgery, University of Minnesota, 420 Delaware St SE, MN, 55455, Minneapolis, USA
| | - David Hu
- Department of Biostatistics, UNC Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 3101 McGavran-Greenberg Hall, CB #7420, Chapel Hill, NC, 27599-7420, USA
| | - Chinmaya U Joisa
- Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill, 10202C Mary Ellen Jones Building, Chapel Hill, NC, 27599, USA
| | - Jose G Guillem
- Divison of Gastrointestinal Surgery, Department of Surgery, University of North Carolina at Chapel Hill, 100 Manning Drive, 4038 Burnett Womack Building, CB #7050, Chapel Hill, NC, 27599, USA
| | - Shawn M Gomez
- Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill, 10202C Mary Ellen Jones Building, Chapel Hill, NC, 27599, USA
| | - Muneera R Kapadia
- Divison of Gastrointestinal Surgery, Department of Surgery, University of North Carolina at Chapel Hill, 100 Manning Drive, 4038 Burnett Womack Building, CB #7050, Chapel Hill, NC, 27599, USA.
| |
Collapse
|
28
|
Ghomrawi HMK, O'Brien MK, Carter M, Macaluso R, Khazanchi R, Fanton M, DeBoer C, Linton SC, Zeineddin S, Pitt JB, Bouchard M, Figueroa A, Kwon S, Holl JL, Jayaraman A, Abdullah F. Applying machine learning to consumer wearable data for the early detection of complications after pediatric appendectomy. NPJ Digit Med 2023; 6:148. [PMID: 37587211 PMCID: PMC10432429 DOI: 10.1038/s41746-023-00890-z] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Accepted: 08/01/2023] [Indexed: 08/18/2023] Open
Abstract
When children are discharged from the hospital after surgery, their caregivers often rely on subjective assessments (e.g., appetite, fatigue) to monitor postoperative recovery as objective assessment tools are scarce at home. Such imprecise and one-dimensional evaluations can result in unwarranted emergency department visits or delayed care. To address this gap in postoperative monitoring, we evaluated the ability of a consumer-grade wearable device, Fitbit, which records multimodal data about daily physical activity, heart rate, and sleep, in detecting abnormal recovery early in children recovering after appendectomy. One hundred and sixty-two children, ages 3-17 years old, who underwent an appendectomy (86 complicated and 76 simple cases of appendicitis) wore a Fitbit device on their wrist for 21 days postoperatively. Abnormal recovery events (i.e., abnormal symptoms or confirmed postoperative complications) that arose during this period were gathered from medical records and patient reports. Fitbit-derived measures, as well as demographic and clinical characteristics, were used to train machine learning models to retrospectively detect abnormal recovery in the two days leading up to the event for patients with complicated and simple appendicitis. A balanced random forest classifier accurately detected 83% of these abnormal recovery days in complicated appendicitis and 70% of abnormal recovery days in simple appendicitis prior to the true report of a symptom/complication. These results support the development of machine learning algorithms to predict onset of abnormal symptoms and complications in children undergoing surgery, and the use of consumer wearables as monitoring tools for early detection of postoperative events.
Collapse
Affiliation(s)
- Hassan M K Ghomrawi
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Center for Health Services and Outcomes Research, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Center for Global Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Department of Medicine (Rheumatology), Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | | | - Michela Carter
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Division of Pediatric Surgery, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | | | - Rushmin Khazanchi
- Shirley Ryan AbilityLab, Chicago, IL, USA
- Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | | | - Christopher DeBoer
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Division of Pediatric Surgery, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Samuel C Linton
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Division of Pediatric Surgery, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Suhail Zeineddin
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Division of Pediatric Surgery, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - J Benjamin Pitt
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Division of Pediatric Surgery, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Megan Bouchard
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Division of Pediatric Surgery, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Angie Figueroa
- Division of Pediatric Surgery, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Soyang Kwon
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Department of Pediatrics, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Jane L Holl
- Department of Neurology and Center for Healthcare Delivery Science and Innovation, Biological Sciences Division, University of Chicago, Chicago, IL, USA
| | - Arun Jayaraman
- Shirley Ryan AbilityLab, Chicago, IL, USA
- Department of Physical Medicine and Rehabilitation, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Department of Physical Therapy and Human Movement Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Fizan Abdullah
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
- Center for Global Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
- Division of Pediatric Surgery, Ann and Robert H. Lurie Children's Hospital of Chicago, 225 East Chicago Avenue, Box 63, Chicago, IL, 60611, USA.
| |
Collapse
|
29
|
Wei R, Guan X, Liu E, Zhang W, Lv J, Huang H, Zhao Z, Chen H, Liu Z, Jiang Z, Wang X. Development of a machine learning algorithm to predict complications of total laparoscopic anterior resection and natural orifice specimen extraction surgery in rectal cancer. EUROPEAN JOURNAL OF SURGICAL ONCOLOGY 2023; 49:1258-1268. [PMID: 36653246 DOI: 10.1016/j.ejso.2023.01.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 11/01/2022] [Accepted: 01/08/2023] [Indexed: 01/11/2023]
Abstract
BACKGROUND Total laparoscopic anterior resection (tLAR) and natural orifice specimen extraction surgery (NOSES) has been widely adopted in the treatment of rectal cancer (RC). However, no study has been performed to predict the short-term outcomes of tLAR using machine learning algorithms to analyze a national cohort. METHODS Data from consecutive RC patients who underwent tLAR were collected from the China NOSES Database (CNDB). The random forest (RF), extreme gradient boosting (XGBoost), support vector machine (SVM), deep neural network (DNN), logistic regression (LR) and K-nearest neighbor (KNN) algorithms were used to develop risk models to predict short-term complications of tLAR. The area under the receiver operating characteristic curve (AUROC), Gini coefficient, specificity and sensitivity were calculated to assess the performance of each risk model. The selected factors from the models were evaluated by relative importance. RESULTS A total of 4313 RC patients were identified, and 667 patients (15.5%) developed postoperative complications. The machine learning model of XGBoost showed more promising results in the prediction of complication than other models (AUROC 0.90, P < 0.001). The performance was similar when internal and external validation was used. In the XGBoost model, the top four influential factors were the distance from the lower edge of the tumor to the anus, age at diagnosis, surgical time and comorbidities. In risk stratification analysis, the rate of postoperative complications in the high-risk group was significantly higher than in the medium- and low-risk groups (P < 0.001). CONCLUSION The machine learning model shows potential benefits in predicting the risk of complications in RC patients after tLAR. This novel approach can provide reliable individual information for surgical treatment recommendations.
Collapse
Affiliation(s)
- Ran Wei
- Department of Colorectal Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Xu Guan
- Department of Colorectal Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Enrui Liu
- Department of Colorectal Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Weiyuan Zhang
- Department of Colorectal Surgery, The Second Affiliated Hospital of Harbin Medical University, Harbin, China
| | - Jingfang Lv
- Department of Colorectal Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Haiyang Huang
- Department of Colorectal Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Zhixun Zhao
- Department of Colorectal Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Haipeng Chen
- Department of Colorectal Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Zheng Liu
- Department of Colorectal Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Zheng Jiang
- Department of Colorectal Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.
| | - Xishan Wang
- Department of Colorectal Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.
| |
Collapse
|
30
|
Tsutsumi K, Goshtasbi K, Ahmed KH, Khosravi P, Tawk K, Haidar YM, Tjoa T, Armstrong WB, Abouzari M. Artificial neural network prediction of post-thyroidectomy outcome. Clin Otolaryngol 2023; 48:665-671. [PMID: 37096572 PMCID: PMC10330281 DOI: 10.1111/coa.14066] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Revised: 02/13/2023] [Accepted: 04/09/2023] [Indexed: 04/26/2023]
Abstract
OBJECTIVES The goal of this study was to develop a deep neural network (DNN) for predicting surgical/medical complications and unplanned reoperations following thyroidectomy. DESIGN, SETTING, AND PARTICIPANTS The 2005-2017 American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) database was queried to extract patients who underwent thyroidectomy. A DNN consisting of 10 layers was developed with an 80:20 breakdown for training and testing. MAIN OUTCOME MEASURES Three primary outcomes of interest, including occurrence of surgical complications, medical complications, and unplanned reoperation were predicted. RESULTS Of the 21 550 patients who underwent thyroidectomy, medical complications, surgical complications and reoperation occurred in 1723 (8.0%), 943 (4.38%) and 2448 (11.36%) patients, respectively. The DNN performed with an area under the curve of receiver operating characteristics of .783 (medical complications), .709 (surgical complications) and .703 (reoperations). Accuracy, specificity and negative predictive values of the model for all outcome variables ranged 78.2%-97.2%, while sensitivity and positive predictive values ranged 11.6%-62.5%. Variables with high permutation importance included sex, inpatient versus outpatient and American Society of Anesthesiologists class. CONCLUSIONS We predicted surgical/medical complications and unplanned reoperation following thyroidectomy via development of a well-performing ML algorithm. We have also developed a web-based application available on mobile devices to demonstrate the predictive capacity of our models in real time.
Collapse
Affiliation(s)
- Kotaro Tsutsumi
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Khodayar Goshtasbi
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Khwaja H. Ahmed
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Pooya Khosravi
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Karen Tawk
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Yarah M. Haidar
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Tjoson Tjoa
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - William B. Armstrong
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Mehdi Abouzari
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| |
Collapse
|
31
|
Taha A, Taha-Mehlitz S, Ortlieb N, Ochs V, Honaker MD, Rosenberg R, Lock JF, Bolli M, Cattin PC. Machine learning in pancreas surgery, what is new? literature review. Front Surg 2023; 10:1142585. [PMID: 37383385 PMCID: PMC10293756 DOI: 10.3389/fsurg.2023.1142585] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Accepted: 05/19/2023] [Indexed: 06/30/2023] Open
Abstract
Background Machine learning (ML) is an inquiry domain that aims to establish methodologies that leverage information to enhance performance of various applications. In the healthcare domain, the ML concept has gained prominence over the years. As a result, the adoption of ML algorithms has become expansive. The aim of this scoping review is to evaluate the application of ML in pancreatic surgery. Methods We integrated the preferred reporting items for systematic reviews and meta-analyses for scoping reviews. Articles that contained relevant data specializing in ML in pancreas surgery were included. Results A search of the following four databases PubMed, Cochrane, EMBASE, and IEEE and files adopted from Google and Google Scholar was 21. The main features of included studies revolved around the year of publication, the country, and the type of article. Additionally, all the included articles were published within January 2019 to May 2022. Conclusion The integration of ML in pancreas surgery has gained much attention in previous years. The outcomes derived from this study indicate an extensive literature gap on the topic despite efforts by various researchers. Hence, future studies exploring how pancreas surgeons can apply different learning algorithms to perform essential practices may ultimately improve patient outcomes.
Collapse
Affiliation(s)
- Anas Taha
- Department of Biomedical Engineering, Faculty of Medicine, University of Basel, Allschwil, Switzerland
| | - Stephanie Taha-Mehlitz
- Clarunis, Department of Visceral Surgery, University Center for Gastrointestinal and Liver Diseases, St. Clara Hospital and University Hospital, Basel, Switzerland
| | - Niklas Ortlieb
- Goethe University Frankfurt, Faculty of Business and Economics, Frankfurt am Main, Germany
| | - Vincent Ochs
- Department of Biomedical Engineering, Faculty of Medicine, University of Basel, Allschwil, Switzerland
| | - Michael Drew Honaker
- Department of Surgery, East Carolina University, Brody School of Medicine, Greenville, NC, United States
| | - Robert Rosenberg
- Cantonal Hospital Basel-Landschaft, Centre for Gastrointestinal and Liver Diseases, Liestal, Switzerland
| | - Johan F. Lock
- Department of General, Visceral, Transplantation, Vascular and Pediatric Surgery, University Hospital Würzburg, Würzburg, Germany
| | - Martin Bolli
- Clarunis, Department of Visceral Surgery, University Center for Gastrointestinal and Liver Diseases, St. Clara Hospital and University Hospital, Basel, Switzerland
| | - Philippe C. Cattin
- Department of Biomedical Engineering, Faculty of Medicine, University of Basel, Allschwil, Switzerland
| |
Collapse
|
32
|
Gendler S, Talmy T, Shapiro M, Tzur D, Kedem R, Landau R, Zubkov K. Predictors of combat training attrition in Israel Defense Forces soldiers. Occup Med (Lond) 2023; 73:80-84. [PMID: 36719096 DOI: 10.1093/occmed/kqad008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/01/2023] Open
Abstract
BACKGROUND Attrition from combat service carries significant organizational and personal ramifications, but predicting factors associated with attrition remains challenging. AIMS To evaluate medical and psychosocial factors associated with attrition from basic combat training (BCT) in the Israel Defense Forces (IDF). In addition, we identify subsets of the recruit population which exhibit certain trends in terms of medical corresponding with a high risk of attrition. METHODS A cross-sectional study of IDF combat trainees undergoing infantry BCT between 2012 and 2017. Data were collected from the soldiers' electronic medical and administrative records. We used multivariable logistic regression and the SAS® decision-tree tool to analyse key predictive factors for attrition. RESULTS A total of 46 472 soldiers enlisted to BCT during the research period. The mean body mass index (BMI) was 21.8 (SD 3.54). The overall attrition rate was 10%. The following factors were associated with attrition from BCT: ethnicity (P < 0.01), BMI (P < 0.01), pre-enlisting motivation score (P < 0.01) and the number of mental health officer visits (P < 0.01). Using a decision-tree model, we found a high attrition rate among soldiers who had >5.2 to primary care physician visits (11% attrition rate versus 3%) or more than 11 sick leave days (59% versus 19%). CONCLUSIONS This study sheds light on unique measures relating to attrition. Attrition is associated with several demographic and psychosocial factors. Early prediction of motivation and monitoring of healthcare utilization may enable early identification and focused interventions targeting soldiers at high risk for attrition. These findings need to be further translated into actionable directives and further investigations.
Collapse
Affiliation(s)
- S Gendler
- Israel Defense Forces Medical Corps, Surgeon General's Headquarters, Ramat Gan 5262000, Israel
| | - T Talmy
- Israel Defense Forces Medical Corps, Surgeon General's Headquarters, Ramat Gan 5262000, Israel
| | - M Shapiro
- Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv 69978, Israel
- Department of Internal Medicine T, Tel Aviv Sourasky Medical Center, Tel Aviv 6423906, Israel
| | - D Tzur
- Medical Information Department, Surgeon General's Headquarters, Medical Corps, Israel Defense Forces, Tel-Hashomer 02149, Israel
| | - R Kedem
- Medical Information Department, Surgeon General's Headquarters, Medical Corps, Israel Defense Forces, Tel-Hashomer 02149, Israel
| | - R Landau
- Israel Defense Forces Medical Corps, Surgeon General's Headquarters, Ramat Gan 5262000, Israel
| | - K Zubkov
- Israel Defense Forces Medical Corps, Surgeon General's Headquarters, Ramat Gan 5262000, Israel
- Maccabi Healthcare Services, Tel Aviv 6812509, Israel
| |
Collapse
|
33
|
Viderman D, Abdildin YG, Batkuldinova K, Badenes R, Bilotta F. Artificial Intelligence in Resuscitation: A Scoping Review. J Clin Med 2023; 12:2254. [PMID: 36983255 PMCID: PMC10054374 DOI: 10.3390/jcm12062254] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 02/17/2023] [Accepted: 03/09/2023] [Indexed: 03/17/2023] Open
Abstract
INTRODUCTION Cardiac arrest is a significant cause of premature mortality and severe disability. Despite the death rate steadily decreasing over the previous decade, only 22% of survivors achieve good clinical status and only 25% of patients survive until their discharge from the hospital. The objective of this scoping review was to review relevant AI modalities and the main potential applications of AI in resuscitation. METHODS We conducted the literature search for related studies in PubMed, EMBASE, and Google Scholar. We included peer-reviewed publications and articles in the press, pooling and characterizing the data by their model types, goals, and benefits. RESULTS After identifying 268 original studies, we chose 59 original studies (reporting 1,817,419 patients) to include in the qualitative synthesis. AI-based methods appear to be superior to traditional methods in achieving high-level performance. CONCLUSION AI might be useful in predicting cardiac arrest, heart rhythm disorders, and post-cardiac arrest outcomes, as well as in the delivery of drone-delivered defibrillators and notification of dispatchers. AI-powered technologies could be valuable assistants to continuously track patient conditions. Healthcare professionals should assist in the research and development of AI-powered technologies as well as their implementation into clinical practice.
Collapse
Affiliation(s)
- Dmitriy Viderman
- Department of Surgery, Nazarbayev University School of Medicine (NUSOM), Kerei, Zhanibek khandar Str. 5/1, Astana 010000, Kazakhstan;
| | - Yerkin G. Abdildin
- Department of Mechanical and Aerospace Engineering, School of Engineering and Digital Sciences, Nazarbayev University, 53 Kabanbay Batyr Ave., Astana 010000, Kazakhstan
| | - Kamila Batkuldinova
- Department of Mechanical and Aerospace Engineering, School of Engineering and Digital Sciences, Nazarbayev University, 53 Kabanbay Batyr Ave., Astana 010000, Kazakhstan
| | - Rafael Badenes
- Department of Anaesthesiology and Intensive Care, Hospital Clìnico Universitario de Valencia, University of Valencia, 46001 Valencia, Spain
| | - Federico Bilotta
- Department of Anesthesia and Intensive Care, University La Sapienza, 00185 Rome, Italy
| |
Collapse
|
34
|
Chen KA, Joisa CU, Stem J, Guillem JG, Eng SMG, Kapadia MR. Improved Prediction of Surgical-Site Infection After Colorectal Surgery Using Machine Learning. Dis Colon Rectum 2023; 66:458-466. [PMID: 36538699 PMCID: PMC10069984 DOI: 10.1097/dcr.0000000000002559] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Abstract
BACKGROUND Surgical-site infection is a source of significant morbidity after colorectal surgery. Previous efforts to develop models that predict surgical-site infection have had limited accuracy. Machine learning has shown promise in predicting postoperative outcomes by identifying nonlinear patterns within large data sets. OBJECTIVE This study aimed to seek usage of machine learning to develop a more accurate predictive model for colorectal surgical-site infections. DESIGN Patients who underwent colorectal surgery were identified in the American College of Surgeons National Quality Improvement Program database from years 2012 to 2019 and were split into training, validation, and test sets. Machine-learning techniques included random forest, gradient boosting, and artificial neural network. A logistic regression model was also created. Model performance was assessed using area under the receiver operating characteristic curve. SETTINGS A national, multicenter data set. PATIENTS Patients who underwent colorectal surgery. MAIN OUTCOME MEASURES The primary outcome (surgical-site infection) included patients who experienced superficial, deep, or organ-space surgical-site infections. RESULTS The data set included 275,152 patients after the application of exclusion criteria. Of all patients, 10.7% experienced a surgical-site infection. Artificial neural network showed the best performance with area under the receiver operating characteristic curve of 0.769 (95% CI, 0.762-0.777), compared with 0.766 (95% CI, 0.759-0.774) for gradient boosting, 0.764 (95% CI, 0.756-0.772) for random forest, and 0.677 (95% CI, 0.669-0.685) for logistic regression. For the artificial neural network model, the strongest predictors of surgical-site infection were organ-space surgical-site infection present at time of surgery, operative time, oral antibiotic bowel preparation, and surgical approach. LIMITATIONS Local institutional validation was not performed. CONCLUSIONS Machine-learning techniques predict colorectal surgical-site infections with higher accuracy than logistic regression. These techniques may be used to identify patients at increased risk and to target preventive interventions for surgical-site infection. See Video Abstract at http://links.lww.com/DCR/C88 . PREDICCIN MEJORADA DE LA INFECCIN DEL SITIO QUIRRGICO DESPUS DE LA CIRUGA COLORRECTAL MEDIANTE EL APRENDIZAJE AUTOMTICO ANTECEDENTES:La infección del sitio quirúrgico es una fuente de morbilidad significativa después de la cirugía colorrectal. Los esfuerzos anteriores para desarrollar modelos que predijeran la infección del sitio quirúrgico han tenido una precisión limitada. El aprendizaje automático se ha mostrado prometedor en la predicción de los resultados posoperatorios mediante la identificación de patrones no lineales dentro de grandes conjuntos de datos.OBJETIVO:Intentamos utilizar el aprendizaje automático para desarrollar un modelo predictivo más preciso para las infecciones del sitio quirúrgico colorrectal.DISEÑO:Los pacientes que se sometieron a cirugía colorrectal se identificaron en la base de datos del Programa Nacional de Mejoramiento de la Calidad del Colegio Estadounidense de Cirujanos de los años 2012 a 2019 y se dividieron en conjuntos de capacitación, validación y prueba. Las técnicas de aprendizaje automático incluyeron conjunto aleatorio, aumento de gradiente y red neuronal artificial. También se creó un modelo de regresión logística. El rendimiento del modelo se evaluó utilizando el área bajo la curva característica operativa del receptor.CONFIGURACIÓN:Un conjunto de datos multicéntrico nacional.PACIENTES:Pacientes intervenidos de cirugía colorrectal.PRINCIPALES MEDIDAS DE RESULTADO:El resultado primario (infección del sitio quirúrgico) incluyó pacientes que experimentaron infecciones superficiales, profundas o del espacio de órganos del sitio quirúrgico.RESULTADOS:El conjunto de datos incluyó 275.152 pacientes después de la aplicación de los criterios de exclusión. El 10,7% de los pacientes presentó infección del sitio quirúrgico. La red neuronal artificial mostró el mejor rendimiento con el área bajo la curva característica operativa del receptor de 0,769 (IC del 95 %: 0,762 - 0,777), en comparación con 0,766 (IC del 95 %: 0,759 - 0,774) para el aumento de gradiente, 0,764 (IC del 95 %: 0,756 - 0,772) para conjunto aleatorio y 0,677 (IC 95% 0,669 - 0,685) para regresión logística. Para el modelo de red neuronal artificial, los predictores más fuertes de infección del sitio quirúrgico fueron la infección del sitio quirúrgico del espacio del órgano presente en el momento de la cirugía, el tiempo operatorio, la preparación intestinal con antibióticos orales y el abordaje quirúrgico.LIMITACIONES:No se realizó validación institucional local.CONCLUSIONES:Las técnicas de aprendizaje automático predicen infecciones del sitio quirúrgico colorrectal con mayor precisión que la regresión logística. Estas técnicas se pueden usar para identificar a los pacientes con mayor riesgo y para orientar las intervenciones preventivas para la infección del sitio quirúrgico. Consulte Video Resumen en http://links.lww.com/DCR/C88 . (Traducción-Dr Yolanda Colorado ).
Collapse
Affiliation(s)
- Kevin A Chen
- Department of Surgery, University of North Carolina, Chapel Hill, NC 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC 27599
| | - Chinmaya U Joisa
- Joint Department of Biomedical Engineering, University of North Carolina, Chapel Hill, NC 10202C Mary Ellen Jones Building, Chapel Hill, NC, 27599
| | - Jonathan Stem
- Department of Surgery, University of North Carolina, Chapel Hill, NC 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC 27599
| | - Jose G Guillem
- Department of Surgery, University of North Carolina, Chapel Hill, NC 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC 27599
| | - Shawn M Gomez Eng
- Joint Department of Biomedical Engineering, University of North Carolina, Chapel Hill, NC 10202C Mary Ellen Jones Building, Chapel Hill, NC, 27599
| | - Muneera R Kapadia
- Department of Surgery, University of North Carolina, Chapel Hill, NC 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC 27599
| |
Collapse
|
35
|
Zeh RD, Konieczkowski D, Shen C, Tsilimigras DI, Kim A, Grignol VP, Contreras CM, Welliver M, Tsung A, Pawlik TM, Pollock RE, Beane JD. Prognostic factors in patients receiving surgery and radiation therapy for retroperitoneal sarcoma: A machine-learning analysis. Surgery 2023; 173:640-644. [PMID: 36369098 DOI: 10.1016/j.surg.2022.08.037] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2022] [Revised: 07/30/2022] [Accepted: 08/24/2022] [Indexed: 11/11/2022]
Abstract
BACKGROUND The addition of radiation therapy to surgery for retroperitoneal sarcoma remains controversial. Improved patient selection may help identify optimal candidates for multimodality treatment. The aim of this analysis was to define prognostic factors among patients who receive radiation therapy and surgery to aid in patient selection for multimodal therapy. METHODS Patients who received radiation therapy and underwent curative-intent resection for retroperitoneal sarcoma between 2004 and 2016 were identified from a national cohort in the United States (National Cancer Database). A machine-based classification and regression tree model was used to generate similar groups of patients relative to overall survival based on preoperative factors. RESULTS A total of 1,443 patients received radiation therapy in addition to surgery. Median age was 61 years old and 55.0% were female. Most patients (66%) received care at an academic or integrated network cancer program. With a median follow-up of 84 months, receipt of radiation therapy was not associated with improved overall survival (P = .81). Classification and regression tree analysis revealed a significant association between overall survival and American Joint Committee on Cancer stage group, age, tumor histology, and Charlson comorbidity score. Application of these parameters via machine learning stratified patients into 5 cohorts with distinct survival outcomes. In the most favorable cohort (Cohort 1: American Joint Committee on Cancer stage group ≤II, age ≤61, histology including fibrosarcoma, well differentiated liposarcoma, myxoid liposarcoma, and leiomyosarcoma), the 5-year overall survival was 81.7% and median overall survival was not reached; in the least favorable cohort (Cohort 6: American Joint Committee on Cancer stage group >II, age >68) where the 5-year survival was 41.3% and median overall survival was 45.2 months (P < .001 versus Cohort 1). CONCLUSION In the absence of a defined survival benefit, patients with advanced American Joint Committee on Cancer stage group, older age, and medical comorbidities have relatively unfavorable overall survival after combined modality therapy and therefore stand the least to gain from the addition of radiation therapy to surgery. In contrast, younger patients with good performance status and retroperitoneal sarcoma histologies with a higher propensity for local recurrence may have the greatest opportunity to benefit from radiation therapy.
Collapse
Affiliation(s)
- Ryan D Zeh
- Department of Surgery, Division of Surgical Oncology, James Cancer Center, The Ohio State University, Columbus, OH; Department of Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
| | - David Konieczkowski
- Department of Radiation Oncology, James Cancer Center, The Ohio State University, Columbus, OH
| | - Chengli Shen
- Department of Surgery, Division of Surgical Oncology, James Cancer Center, The Ohio State University, Columbus, OH
| | - Diamantis I Tsilimigras
- Department of Surgery, Division of Surgical Oncology, James Cancer Center, The Ohio State University, Columbus, OH
| | - Alex Kim
- Department of Surgery, Division of Surgical Oncology, James Cancer Center, The Ohio State University, Columbus, OH
| | - Valerie P Grignol
- Department of Surgery, Division of Surgical Oncology, James Cancer Center, The Ohio State University, Columbus, OH
| | - Carlo M Contreras
- Department of Surgery, Division of Surgical Oncology, James Cancer Center, The Ohio State University, Columbus, OH
| | - Meng Welliver
- Department of Radiation Oncology, James Cancer Center, The Ohio State University, Columbus, OH
| | - Allan Tsung
- Department of Surgery, Division of Surgical Oncology, James Cancer Center, The Ohio State University, Columbus, OH
| | - Timothy M Pawlik
- Department of Surgery, Division of Surgical Oncology, James Cancer Center, The Ohio State University, Columbus, OH
| | - Raphael E Pollock
- Department of Surgery, Division of Surgical Oncology, James Cancer Center, The Ohio State University, Columbus, OH
| | - Joal D Beane
- Department of Surgery, Division of Surgical Oncology, James Cancer Center, The Ohio State University, Columbus, OH.
| |
Collapse
|
36
|
Lin V, Gögenur S, Pachler F, Fransgaard T, Gögenur I. Risk Prediction for Complications in Inflammatory Bowel Disease Surgery: External Validation of the American College of Surgeons' National Surgical Quality Improvement Program Surgical Risk Calculator. J Crohns Colitis 2023; 17:73-82. [PMID: 35973971 DOI: 10.1093/ecco-jcc/jjac114] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Indexed: 01/31/2023]
Abstract
BACKGROUND AND AIMS Many patients with inflammatory bowel disease [IBD] require surgery during their disease course. Having individual risk predictions available prior to surgery could aid in better informed decision making for personalised treatment trajectories in IBD surgery. The American College of Surgeons National Surgical Quality Improvement Program [ACS NSQIP] has developed a surgical risk calculator that calculates risks for postoperative outcomes using 20 patient and surgical predictors. We aimed to validate the calculator for IBD surgery to determine its accuracy in this patient cohort. METHODS Predicted risks were calculated for patients operated for IBD between December 2017 and January 2022 at two tertiary centres and compared with actual outcomes within 30 postoperative days. Predictive performance was assessed for several postoperative complications, using metrics for discrimination and calibration. RESULTS Risks were calculated for 508 patient trajectories undergoing surgery for IBD. Incidence of any complication, serious complications, reoperation, and readmission were 32.1%, 21.1%, 15.2%, and 18.3%, respectively. Of 212 patients with an anastomosis, 19 experienced leakage [9.0%]. Discriminative performance and calibration were modest. Risk prediction for any complication, serious complication, reoperation, readmission, and anastomotic leakage had a c statistic of 0.605 (95% confidence interval [CI] 0.534-0.640), 0.623 [95% CI 0.558-0.688], 0.590 [95% CI 0.513-0.668], 0.621 [95% CI 0.557-0.685], and 0.574 [95% CI 0.396-0.751], respectively, and a Brier score of 0.240, 0.166, 0.138, 0.152, and 0.113, respectively. CONCLUSIONS The accuracy of risks calculated by the ACS NSQIP Surgical Risk Calculator was deemed insufficient for patients undergoing surgery for IBD, generally underestimating postoperative risks. Recalibration or additional variables could be necessary to predict risks in this cohort.
Collapse
Affiliation(s)
- Viviane Lin
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Denmark
| | - Seyma Gögenur
- Department of Surgery, Herlev Hospital, HerlevDenmark
| | | | - Tina Fransgaard
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Denmark.,Department of Surgery, Herlev Hospital, HerlevDenmark
| | - Ismail Gögenur
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Denmark
| |
Collapse
|
37
|
Hassan AM, Rajesh A, Asaad M, Jonas NA, Coert JH, Mehrara BJ, Butler CE. Artificial Intelligence and Machine Learning in Prediction of Surgical Complications: Current State, Applications, and Implications. Am Surg 2023; 89:25-30. [PMID: 35562124 PMCID: PMC9653510 DOI: 10.1177/00031348221101488] [Citation(s) in RCA: 25] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Surgical complications pose significant challenges for surgeons, patients, and health care systems as they may result in patient distress, suboptimal outcomes, and higher health care costs. Artificial intelligence (AI)-driven models have revolutionized the field of surgery by accurately identifying patients at high risk of developing surgical complications and by overcoming several limitations associated with traditional statistics-based risk calculators. This article aims to provide an overview of AI in predicting surgical complications using common machine learning and deep learning algorithms and illustrates how this can be utilized to risk stratify patients preoperatively. This can form the basis for discussions on informed consent based on individualized patient factors in the future.
Collapse
Affiliation(s)
- Abbas M. Hassan
- Department of Plastic and Reconstructive Surgery, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Aashish Rajesh
- Department of Surgery, University of Texas Health Science Center, San Antonio, TX, USA
| | - Malke Asaad
- Department of Plastic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | - Nelson A. Jonas
- Department of Plastic and Reconstructive Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - J. Henk. Coert
- Department of Plastic and Reconstructive Surgery, University Medical Center Utrecht, Utrecht, Netherlands
| | - Babak J. Mehrara
- Department of Plastic and Reconstructive Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Charles E. Butler
- Department of Plastic and Reconstructive Surgery, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| |
Collapse
|
38
|
Lee W, Schwartz N, Bansal A, Khor S, Hammarlund N, Basu A, Devine B. A Scoping Review of the Use of Machine Learning in Health Economics and Outcomes Research: Part 2-Data From Nonwearables. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2022; 25:2053-2061. [PMID: 35989154 DOI: 10.1016/j.jval.2022.07.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Revised: 06/10/2022] [Accepted: 07/10/2022] [Indexed: 06/15/2023]
Abstract
OBJECTIVES Despite the increasing interest in applying machine learning (ML) methods in health economics and outcomes research (HEOR), stakeholders face uncertainties in when and how ML can be used. We reviewed the recent applications of ML in HEOR. METHODS We searched PubMed for studies published between January 2020 and March 2021 and randomly chose 20% of the identified studies for the sake of manageability. Studies that were in HEOR and applied an ML technique were included. Studies related to wearable devices were excluded. We abstracted information on the ML applications, data types, and ML methods and analyzed it using descriptive statistics. RESULTS We retrieved 805 articles, of which 161 (20%) were randomly chosen. Ninety-two of the random sample met the eligibility criteria. We found that ML was primarily used for predicting future events (86%) rather than current events (14%). The most common response variables were clinical events or disease incidence (42%) and treatment outcomes (22%). ML was less used to predict economic outcomes such as health resource utilization (16%) or costs (3%). Although electronic medical records (35%) were frequently used for model development, claims data were used less frequently (9%). Tree-based methods (eg, random forests and boosting) were the most commonly used ML methods (31%). CONCLUSIONS The use of ML techniques in HEOR is growing rapidly, but there remain opportunities to apply them to predict economic outcomes, especially using claims databases, which could inform the development of cost-effectiveness models.
Collapse
Affiliation(s)
- Woojung Lee
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA.
| | - Naomi Schwartz
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Aasthaa Bansal
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Sara Khor
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Noah Hammarlund
- Department of Health Services Research, Management & Policy, University of Florida, Gainesville, FL, USA
| | - Anirban Basu
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Beth Devine
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| |
Collapse
|
39
|
Wu G, Khair S, Yang F, Cheligeer C, Southern D, Zhang Z, Feng Y, Xu Y, Quan H, Williamson T, Eastwood CA. Performance of machine learning algorithms for surgical site infection case detection and prediction: A systematic review and meta-analysis. Ann Med Surg (Lond) 2022; 84:104956. [PMID: 36582918 PMCID: PMC9793260 DOI: 10.1016/j.amsu.2022.104956] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Revised: 11/08/2022] [Accepted: 11/13/2022] [Indexed: 11/24/2022] Open
Abstract
Background Medical researchers and clinicians have shown much interest in developing machine learning (ML) algorithms to detect/predict surgical site infections (SSIs). However, little is known about the overall performance of ML algorithms in predicting SSIs and how to improve the algorithm's robustness. We conducted a systematic review and meta-analysis to summarize the performance of ML algorithms in SSIs case detection and prediction and to describe the impact of using unstructured and textual data in the development of ML algorithms. Methods MEDLINE, EMBASE, CINAHL, CENTRAL and Web of Science were searched from inception to March 25, 2021. Study characteristics and algorithm development information were extracted. Performance statistics (e.g., sensitivity, area under the receiver operating characteristic curve [AUC]) were pooled using a random effect model. Stratified analysis was applied to different study characteristic levels. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Diagnostic Test Accuracy Studies (PRISMA-DTA) was followed. Results Of 945 articles identified, 108 algorithms from 32 articles were included in this review. The overall pooled estimate of the SSI incidence rate was 3.67%, 95% CI: 3.58-3.76. Mixed-use of structured and textual data-based algorithms (pooled estimates of sensitivity 0.83, 95% CI: 0.78-0.87, specificity 0.92, 95% CI: 0.86-0.95, AUC 0.92, 95% CI: 0.89-0.94) outperformed algorithms solely based on structured data (sensitivity 0.56, 95% CI:0.43-0.69, specificity 0.95, 95% CI:0.91-0.97, AUC = 0.90, 95% CI: 0.87-0.92). Conclusions ML algorithms developed with structured and textual data provided optimal performance. External validation of ML algorithms is needed to translate current knowledge into clinical practice.
Collapse
Affiliation(s)
- Guosong Wu
- Centre for Health Informatics, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
- O'Brien Institute for Public Health, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
- Institute of Health Economics, University of Alberta, Edmonton, Alberta, Canada
| | - Shahreen Khair
- Centre for Health Informatics, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Fengjuan Yang
- Centre for Health Informatics, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | | | - Danielle Southern
- Centre for Health Informatics, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
- O'Brien Institute for Public Health, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Zilong Zhang
- Centre for Health Informatics, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Yuanchao Feng
- Centre for Health Informatics, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Yuan Xu
- Centre for Health Informatics, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
- Department of Oncology and Surgery, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Hude Quan
- Centre for Health Informatics, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
- O'Brien Institute for Public Health, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Tyler Williamson
- Centre for Health Informatics, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
- O'Brien Institute for Public Health, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Cathy A. Eastwood
- Centre for Health Informatics, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| |
Collapse
|
40
|
Chen KA, Joisa CU, Stitzenberg KB, Stem J, Guillem JG, Gomez SM, Kapadia MR. Development and Validation of Machine Learning Models to Predict Readmission After Colorectal Surgery. J Gastrointest Surg 2022; 26:2342-2350. [PMID: 36070116 PMCID: PMC10081888 DOI: 10.1007/s11605-022-05443-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Accepted: 08/18/2022] [Indexed: 01/31/2023]
Abstract
BACKGROUND Readmission after colorectal surgery is common and often implies complications for patients and costs for hospitals. Previous works have created predictive models using logistic regression for this outcome but have shown limited accuracy. Machine learning has shown promise in improving predictions by identifying non-linear patterns in data. We sought to create a more accurate predictive model for readmission after colorectal surgery using machine learning. METHODS Patients who underwent colorectal surgery were identified in the National Quality Improvement Program (NSQIP) database including years 2012-2019 and split into training, validation, and test sets. The primary outcome was readmission within 30 days of surgery. Three types of machine learning models were created, including random forest (RF), gradient boosting (XGB), and neural network (NN). A logistic regression (LR) model was also created for comparison. Model performance was evaluated using area under the receiver operating characteristic curve (AUROC). RESULTS The dataset included 213,827 patients after application of exclusion criteria. A total of 23,083 (10.8%) of patients experienced readmission. NN obtained an AUROC of 0.751 (95% CI 0.743-0.759), compared with 0.684 (95% CI 0.676-0.693) for LR. RF and XGB performed similarly with AUROCs of 0.749 (95% CI 0.741-0.757) and 0.745 (95% CI 0.737-0.753) respectively. Ileus, index admission length of stay, organ-space surgical site infection present at time of surgery, and ostomy placement were identified as the most contributory variables. CONCLUSIONS Machine learning approaches outperformed traditional statistical methods in the prediction of readmission after colorectal surgery. After external validation, this improved prediction model could be used to target interventions to reduce readmission rate.
Collapse
Affiliation(s)
- Kevin A Chen
- Department of Surgery, University of North Carolina, 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC, 27599, USA
| | - Chinmaya U Joisa
- Joint Department of Biomedical Engineering, University of North Carolina, 10202C Mary Ellen Jones Building, Chapel Hill, NC, 27599, USA
| | - Karyn B Stitzenberg
- Department of Surgery, University of North Carolina, 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC, 27599, USA
| | - Jonathan Stem
- Department of Surgery, University of North Carolina, 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC, 27599, USA
| | - Jose G Guillem
- Department of Surgery, University of North Carolina, 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC, 27599, USA
| | - Shawn M Gomez
- Joint Department of Biomedical Engineering, University of North Carolina, 10202C Mary Ellen Jones Building, Chapel Hill, NC, 27599, USA
| | - Muneera R Kapadia
- Department of Surgery, University of North Carolina, 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC, 27599, USA.
| |
Collapse
|
41
|
Ruan X, Fu S, Storlie CB, Mathis KL, Larson DW, Liu H. Real-time risk prediction of colorectal surgery-related post-surgical complications using GRU-D model. J Biomed Inform 2022; 135:104202. [PMID: 36162805 DOI: 10.1016/j.jbi.2022.104202] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Revised: 08/21/2022] [Accepted: 09/04/2022] [Indexed: 10/31/2022]
Abstract
BACKGROUND Post-surgical complications (PSCs) have been an increasing concern for hospitals in light of Medicare penalties for 30-day readmissions. PSCs have become a target for quality improvement and benchmark for the healthcare system. Over half (60 %) of the deep or organ space surgical site infections are discovered after discharge, leading to a readmission. There has thus been a push to develop risk prediction models for targeted preventive interventions for PSCs. OBJECTIVE We experiment several Gated Recurrent Unit with Decay (GRU-D) based deep learning architectures with various feature sampling schemes in predicting the risk of colorectal PSCs and compare with atemporal logistic regression models (logit). METHOD We used electronic health record (EHR) data of 3,535 colorectal surgical patients involved in the national surgical quality improvement program (NSQIP) between 2006 and 2018. Single layer, stacked layer, and multimodal GRU-D models with sigmoid activation were used to develop risk prediction models. Area Under the Receiver Operating Characteristic curve (AUROC) was calculated by comparing predicted probability of developing complications versus truly observed PSCs (NSQIP adjudicated) within 30 days after surgery. We set up cross-validation and an independent held-out dataset for testing model performance consistency. RESULTS AND CONCLUSION The primary contribution of our study is the formulation of a novel real-time PSC risk prediction task using GRU-D with demonstrated clinical utility. GRU-D outperforms the logit model in predicting wound and organ space infection and shows improved performance as additional data points become available. Logit model outperforms GRU-D before surgery for superficial infection and bleeding. For the same sampling scheme, there is no obvious advantage of complex architectures (stacked, multimodal) over single layer GRU-D. Obtaining more data points closer to the occurrence of PSCs is more important than using a more frequent sampling scheme in training GRU-D models. The fourth predicted risk quartile by single layer GRU-D contains 63 %, 59 %, and 66 % organ space infection cases, at 4 h before, 72 h after, and 168 h after surgery, respectively, suggesting its potential application as a bedside risk assessment tool.
Collapse
Affiliation(s)
- Xiaoyang Ruan
- Department of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, United States
| | - Sunyang Fu
- Department of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, United States
| | - Curtis B Storlie
- Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, United States
| | - Kellie L Mathis
- Department of Surgery, Mayo Clinic, Rochester, MN, United States
| | - David W Larson
- Department of Surgery, Mayo Clinic, Rochester, MN, United States
| | - Hongfang Liu
- Department of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, United States.
| |
Collapse
|
42
|
Cotter G, Beal EW, Poultsides GA, Idrees K, Fields RC, Weber SM, Scoggins CR, Shen P, Wolfgang C, Maithel SK, Pawlik TM. Using machine learning to preoperatively stratify prognosis among patients with gallbladder cancer: a multi-institutional analysis. HPB (Oxford) 2022; 24:1980-1988. [PMID: 35798655 DOI: 10.1016/j.hpb.2022.06.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/26/2021] [Revised: 02/13/2022] [Accepted: 06/15/2022] [Indexed: 12/12/2022]
Abstract
BACKGROUND Gallbladder cancer (GBC) is an aggressive malignancy associated with a high risk of recurrence and mortality. We used a machine-based learning approach to stratify patients into distinct prognostic groups using preperative variables. METHODS Patients undergoing curative-intent resection of GBC were identified using a multi-institutional database. A classification and regression tree (CART) was used to stratify patients relative to overall survival (OS) based on preoperative clinical factors. RESULTS CART analysis identified tumor size, biliary drainage, carbohydrate antigen 19-9 (CA19-9) levels, and neutrophil-lymphocyte ratio (NLR) as the factors most strongly associated with OS. Machine learning cohorted patients into four prognostic groups: Group 1 (n = 109): NLR ≤1.5, CA19-9 ≤20, no drainage, tumor size <5.0 cm; Group 2 (n = 88): NLR >1.5, CA19-9 ≤20, no drainage, tumor size <5.0 cm; Group 3 (n = 46): CA19-9 >20, no drainage, tumor size <5.0 cm; Group 4 (n = 77): tumor size <5.0 cm with drainage OR tumor size ≥5.0 cm. Median OS decreased incrementally with CART group designation (59.5, 27.6, 20.6, and 12.1 months; p < 0.0001). CONCLUSIONS A machine-based model was able to stratify GBC patients into four distinct prognostic groups based only on preoperative characteristics. Characterizing patient prognosis with machine learning tools may help physicians provide more patient-centered care.
Collapse
Affiliation(s)
- Garrett Cotter
- Division of Surgical Oncology, The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
| | - Eliza W Beal
- Division of Surgical Oncology, The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
| | - George A Poultsides
- Department of Surgery, Stanford University Medical Center, Stanford, CA, USA
| | - Kamran Idrees
- Division of Surgical Oncology, Department of Surgery, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Ryan C Fields
- Department of Surgery, Washington University School of Medicine, St Louis, MO, USA
| | - Sharon M Weber
- Department of Surgery, University of Wisconsin School of Medicine and Public Health, Madison, WI, USA
| | - Charles R Scoggins
- Division of Surgical Oncology, Department of Surgery, University of Louisville, Louisville, KY, USA
| | - Perry Shen
- Department of Surgery, Wake Forest University, Winston-Salem, NC, USA
| | | | - Shishir K Maithel
- Division of Surgical Oncology, Department of Surgery, Winship Cancer Institute, Emory University, Atlanta, GA, USA
| | - Timothy M Pawlik
- Division of Surgical Oncology, The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA.
| |
Collapse
|
43
|
Peng X, Zhu T, Wang T, Wang F, Li K, Hao X. Machine learning prediction of postoperative major adverse cardiovascular events in geriatric patients: a prospective cohort study. BMC Anesthesiol 2022; 22:284. [PMID: 36088288 PMCID: PMC9463850 DOI: 10.1186/s12871-022-01827-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 08/26/2022] [Indexed: 12/05/2022] Open
Abstract
Background Postoperative major adverse cardiovascular events (MACEs) account for more than one-third of perioperative deaths. Geriatric patients are more vulnerable to postoperative MACEs than younger patients. Identifying high-risk patients in advance can help with clinical decision making and improve prognosis. This study aimed to develop a machine learning model for the preoperative prediction of postoperative MACEs in geriatric patients. Methods We collected patients’ clinical data and laboratory tests prospectively. All patients over 65 years who underwent surgeries in West China Hospital of Sichuan University from June 25, 2019 to June 29, 2020 were included. Models based on extreme gradient boosting (XGB), gradient boosting machine, random forest, support vector machine, and Elastic Net logistic regression were trained. The models’ performance was compared according to area under the precision-recall curve (AUPRC), area under the receiver operating characteristic curve (AUROC) and Brier score. To minimize the influence of clinical intervention, we trained the model based on undersampling set. Variables with little contribution were excluded to simplify the model for ensuring the ease of use in clinical settings. Results We enrolled 5705 geriatric patients into the final dataset. Of those patients, 171 (3.0%) developed postoperative MACEs within 30 days after surgery. The XGB model outperformed other machine learning models with AUPRC of 0.404(95% confidence interval [CI]: 0.219–0.589), AUROC of 0.870(95%CI: 0.786–0.938) and Brier score of 0.024(95% CI: 0.016–0.032). Model trained on undersampling set showed improved performance with AUPRC of 0.511(95% CI: 0.344–0.667, p < 0.001), AUROC of 0.912(95% CI: 0.847–0.962, p < 0.001) and Brier score of 0.020 (95% CI: 0.013–0.028, p < 0.001). After removing variables with little contribution, the undersampling model showed comparable predictive accuracy with AUPRC of 0.507(95% CI: 0.338–0.669, p = 0.36), AUROC of 0.896(95%CI: 0.826–0.953, p < 0.001) and Brier score of 0.020(95% CI: 0.013–0.028, p = 0.20). Conclusions In this prospective study, we developed machine learning models for preoperative prediction of postoperative MACEs in geriatric patients. The XGB model showed the best performance. Undersampling method achieved further improvement of model performance. Trial registration The protocol of this study was registered at www.chictr.org.cn (15/08/2019, ChiCTR1900025160) Supplementary Information The online version contains supplementary material available at 10.1186/s12871-022-01827-x.
Collapse
|
44
|
Onishchenko D, Rubin DS, van Horne JR, Ward RP, Chattopadhyay I. Cardiac Comorbidity Risk Score: Zero-Burden Machine Learning to Improve Prediction of Postoperative Major Adverse Cardiac Events in Hip and Knee Arthroplasty. J Am Heart Assoc 2022; 11:e023745. [PMID: 35904198 PMCID: PMC9375497 DOI: 10.1161/jaha.121.023745] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background In this retrospective, observational study we introduce the Cardiac Comorbidity Risk Score, predicting perioperative major adverse cardiac events (MACE) after elective hip and knee arthroplasty. MACE is a rare but important driver of mortality, and existing tools, eg, the Revised Cardiac Risk Index demonstrate only modest accuracy. We demonstrate an artificial intelligence-based approach to identify patients at high risk of MACE within 4 weeks (primary outcome) of arthroplasty, that imposes zero additional burden of cost/resources. Methods and Results Cardiac Comorbidity Risk Score calculation uses novel machine learning to estimate MACE risk from patient electronic health records, without requiring blood work or access to any demographic data beyond that of sex and age, and accounts for variable/missing/incomplete information across patient records. Validated on a deidentified cohort (age >45 years, n=445 391), performance was evaluated using the area under the receiver operator characteristics curve (AUROC), sensitivity/specificity, positive predictive value, and positive/negative likelihood ratios. In our cohort (age 63.5±10.5 years, 58.2% women, 34.2%/65.8% hip/knee procedures), 0.19% (882) experienced the primary outcome. Cardiac Comorbidity Risk Score achieved area under the receiver operator characteristics curve=80.0±0.4% (95% CI) for women and 80.1±0.5% (95% CI) for males, with 36.4% and 35.1% sensitivities, respectively, at 95% specificity, significantly outperforming Revised Cardiac Risk Index across all studied age-, sex-, risk-, and comorbidity-based subgroups. Conclusions Cardiac Comorbidity Risk Score, a novel artificial intelligence-based screening tool using known and unknown comorbidity patterns, outperforms state-of-the-art in predicting MACE within 4 weeks postarthroplasty, and can identify patients at high risk that do not demonstrate traditional risk factors.
Collapse
Affiliation(s)
| | - Daniel S Rubin
- Department of Anesthesia and Critical Care University of Chicago IL
| | | | - R Parker Ward
- Department of Medicine University of Chicago IL.,Section of Cardiology University of Chicago IL
| | - Ishanu Chattopadhyay
- Department of Medicine University of Chicago IL.,Committee on Genetics, Genomics & Systems Biology University of Chicago IL.,Committee on Quantitative Methods in Social, Behavioral, and Health Sciences University of Chicago IL.,Section of Hospital Medicine University of Chicago IL
| |
Collapse
|
45
|
Chen KA, Berginski ME, Desai CS, Guillem JG, Stem J, Gomez Eng SM, Kapadia MR. Differential Performance of Machine Learning Models in Prediction of Procedure-Specific Outcomes. J Gastrointest Surg 2022; 26:1732-1742. [PMID: 35508684 PMCID: PMC9444966 DOI: 10.1007/s11605-022-05332-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Accepted: 04/02/2022] [Indexed: 01/31/2023]
Abstract
BACKGROUND Procedure-specific complications can have devastating consequences. Machine learning-based tools have the potential to outperform traditional statistical modeling in predicting their risk and guiding decision-making. We sought to develop and compare deep neural network (NN) models, a type of machine learning, to logistic regression (LR) for predicting anastomotic leak after colectomy, bile leak after hepatectomy, and pancreatic fistula after pancreaticoduodenectomy (PD). METHODS The colectomy, hepatectomy, and PD National Surgical Quality Improvement Program (NSQIP) databases were analyzed. Each dataset was split into training, validation, and testing sets in a 60/20/20 ratio, with fivefold cross-validation. Models were created using NN and LR for each outcome. Models were evaluated primarily with area under the receiver operating characteristic curve (AUROC). RESULTS A total of 197,488 patients were included for colectomy, 25,403 for hepatectomy, and 23,333 for PD. For anastomotic leak, AUROC for NN was 0.676 (95% 0.666-0.687), compared with 0.633 (95% CI 0.620-0.647) for LR. For bile leak, AUROC for NN was 0.750 (95% CI 0.739-0.761), compared with 0.722 (95% CI 0.698-0.746) for LR. For pancreatic fistula, AUROC for NN was 0.746 (95% CI 0.733-0.760), compared with 0.713 (95% CI 0.703-0.723) for LR. Variables related to intra-operative information, such as surgical approach, biliary reconstruction, and pancreatic gland texture were highly important for model predictions. DISCUSSION Machine learning showed a marginal advantage over traditional statistical techniques in predicting procedure-specific outcomes. However, models that included intra-operative information performed better than those that did not, suggesting that NSQIP procedure-targeted datasets may be strengthened by including relevant intra-operative information.
Collapse
Affiliation(s)
- Kevin A Chen
- Department of Surgery, University of North Carolina, Chapel Hill, NC, 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC 27599
| | - Matthew E Berginski
- Department of Pharmacology, University of North Carolina, Chapel Hill, NC, 120 Mason Farm Rd, Genetic Medicine Building, Chapel Hill, NC 27599
| | - Chirag S Desai
- Department of Surgery, University of North Carolina, Chapel Hill, NC, 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC 27599
| | - Jose G Guillem
- Department of Surgery, University of North Carolina, Chapel Hill, NC, 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC 27599
| | - Jonathan Stem
- Department of Surgery, University of North Carolina, Chapel Hill, NC, 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC 27599
| | - Shawn M Gomez Eng
- Department of Pharmacology, University of North Carolina, Chapel Hill, NC, 120 Mason Farm Rd, Genetic Medicine Building, Chapel Hill, NC 27599,Joint Department of Biomedical Engineering, University of North Carolina, Chapel Hill, NC, 10202C Mary Ellen Jones Building, Chapel Hill, NC, 27599
| | - Muneera R Kapadia
- Department of Surgery, University of North Carolina, Chapel Hill, NC, 100 Manning Drive, Burnett Womack Building, Suite 4038, Chapel Hill, NC 27599
| |
Collapse
|
46
|
Mochão H, Gonçalves D, Alexandre L, Castro C, Valério D, Barahona P, Moreira-Gonçalves D, Costa PMD, Henriques R, Santos LL, Costa RS. IPOscore: An interactive web-based platform for postoperative surgical complications analysis and prediction in the oncology domain. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 219:106754. [PMID: 35364482 DOI: 10.1016/j.cmpb.2022.106754] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/12/2021] [Revised: 03/07/2022] [Accepted: 03/11/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND The performance of traditional risk score systems to predict (post)-operative outcomes is limited. This weakness reduces confidence in its use to support clinical risk mitigation decisions. However, the rapid growth of health data in the last years offers principles to deal with some of these limitations. In this regard, the data allows the extraction of relevant information for both patients stratification and the rigorous identification of associated risk factors. The patients can then be targeted to specific preoperative optimization programs, thus contributing to the reduction of associated morbidity and mortality. OBJECTIVES The main goal of this work is, therefore, to provide a clinical decision support system (CDSS) based on data-driven modeling methods for surgical risk prediction specific for cancer patients in Portugal. RESULTS The result is IPOscore, a single web-based platform aimed at being an innovative approach to assist clinical decision-making in the surgical oncology domain. This system includes a database to store/manage the clinical data collected in a structured format, data visualization and analysis tools, and predictive machine learning models to predict postoperative outcomes in cancer patients. IPOscore also includes a pattern mining module based on biclustering to assess the discriminative power of a pattern towards postsurgical outcomes. Additionally, a mobile application is provided to this end. CONCLUSIONS The IPOscore platform is a valuable tool for surgical oncologists not only for clinical data management but also as a preventative and predictive healthcare system. Currently, this clinical support tool is being tested at the Portuguese Institute of Oncology (IPO-Porto), and can be accessed online at https://iposcore.org.
Collapse
Affiliation(s)
- Hugo Mochão
- IDMEC, Instituto Superior Tecnico, Universidade de Lisboa, Lisboa, Av. Rovisco Pais 1, Lisboa, 1049-001, Portugal
| | - Daniel Gonçalves
- IDMEC, Instituto Superior Tecnico, Universidade de Lisboa, Lisboa, Av. Rovisco Pais 1, Lisboa, 1049-001, Portugal; LAQV-REQUIMTE, DQ, NOVA School of Science and Technology, Universidade NOVA de Lisboa, Campus da Caparica, Caparica, 2829-516, Portugal; INESC-ID, Lisboa, Portugal, R. Alves Redol 9, Lisboa, 1000-029, Portugal
| | - Leonardo Alexandre
- IDMEC, Instituto Superior Tecnico, Universidade de Lisboa, Lisboa, Av. Rovisco Pais 1, Lisboa, 1049-001, Portugal; LAQV-REQUIMTE, DQ, NOVA School of Science and Technology, Universidade NOVA de Lisboa, Campus da Caparica, Caparica, 2829-516, Portugal; INESC-ID, Lisboa, Portugal, R. Alves Redol 9, Lisboa, 1000-029, Portugal
| | - Carolina Castro
- Experimental Pathology and Therapeutics Group of Portuguese Institute of Oncology of Porto FG, EPE (IPO-Porto), Porto, Portugal
| | - Duarte Valério
- IDMEC, Instituto Superior Tecnico, Universidade de Lisboa, Lisboa, Av. Rovisco Pais 1, Lisboa, 1049-001, Portugal
| | - Pedro Barahona
- NOVA LINCS, Dept. Informatica Faculdade de Ciencias e Tecnologia, Universidade NOVA de Lisboa, Campus da Caparica, Caparica, 2829-516, Portugal
| | - Daniel Moreira-Gonçalves
- Research Centre in Physical Activity, Health and Leisure, Faculdade de Desporto, Universidade do Porto, Porto, Portugal
| | - Paulo Matos da Costa
- General Surgery Service, Hospital Garcia de Orta, E.P.E., Portugal; Faculdade de Medicina da Universidade de Lisboa, Portugal
| | - Rui Henriques
- INESC-ID, Lisboa, Portugal, R. Alves Redol 9, Lisboa, 1000-029, Portugal; Instituto Superior Tecnico, University of Lisbon, Lisbon, Portugal
| | - Lúcio L Santos
- Surgical ICU of the Portuguese Institute of Oncology, Porto, Portugal; Surgical Oncology Department, IPO-Porto, Porto, Portugal; Experimental Pathology and Therapeutics Group of Portuguese Institute of Oncology of Porto FG, EPE (IPO-Porto), Porto, Portugal
| | - Rafael S Costa
- LAQV-REQUIMTE, DQ, NOVA School of Science and Technology, Universidade NOVA de Lisboa, Campus da Caparica, Caparica, 2829-516, Portugal; IDMEC, Instituto Superior Tecnico, Universidade de Lisboa, Lisboa, Av. Rovisco Pais 1, Lisboa, 1049-001, Portugal.
| |
Collapse
|
47
|
Lin V, Tsouchnika A, Allakhverdiiev E, Rosen AW, Gögenur M, Clausen JSR, Bräuner KB, Walbech JS, Rijnbeek P, Drakos I, Gögenur I. Training prediction models for individual risk assessment of postoperative complications after surgery for colorectal cancer. Tech Coloproctol 2022; 26:665-675. [PMID: 35593971 DOI: 10.1007/s10151-022-02624-x] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/13/2022] [Accepted: 04/20/2022] [Indexed: 12/01/2022]
Abstract
BACKGROUND The occurrence of postoperative complications and anastomotic leakage are major drivers of mortality in the immediate phase after colorectal cancer surgery. We trained prediction models for calculating patients' individual risk of complications based only on preoperatively available data in a multidisciplinary team setting. Knowing prior to surgery the probability of developing a complication could aid in improving informed decision-making by surgeon and patient and individualize surgical treatment trajectories. METHODS All patients over 18 years of age undergoing any resection for colorectal cancer between January 1, 2014 and December 31, 2019 from the nationwide Danish Colorectal Cancer Group database were included. Data from the database were converted into Observational Medical Outcomes Partnership Common Data Model maintained by the Observation Health Data Science and Informatics initiative. Multiple machine learning models were trained to predict postoperative complications of Clavien-Dindo grade ≥ 3B and anastomotic leakage within 30 days after surgery. RESULTS Between 2014 and 2019, 23,907 patients underwent resection for colorectal cancer in Denmark. A Clavien-Dindo complication grade ≥ 3B occurred in 2,958 patients (12.4%). Of 17,190 patients that received an anastomosis, 929 experienced anastomotic leakage (5.4%). Among the compared machine learning models, Lasso Logistic Regression performed best. The predictive model for complications had an area under the receiver operating characteristic curve (AUROC) of 0.704 (95%CI 0.683-0.724) and an AUROC of 0.690 (95%CI 0.655-0.724) for anastomotic leakage. CONCLUSIONS The prediction of postoperative complications based only on preoperative variables using a national quality assurance colorectal cancer database shows promise for calculating patient's individual risk. Future work will focus on assessing the value of adding laboratory parameters and drug exposure as candidate predictors. Furthermore, we plan to assess the external validity of our proposed model.
Collapse
Affiliation(s)
- V Lin
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Lykkebækvej 1, 4600, Køge, Denmark.
| | - A Tsouchnika
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Lykkebækvej 1, 4600, Køge, Denmark
| | - E Allakhverdiiev
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Lykkebækvej 1, 4600, Køge, Denmark
| | - A W Rosen
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Lykkebækvej 1, 4600, Køge, Denmark
| | - M Gögenur
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Lykkebækvej 1, 4600, Køge, Denmark
| | - J S R Clausen
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Lykkebækvej 1, 4600, Køge, Denmark
| | - K B Bräuner
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Lykkebækvej 1, 4600, Køge, Denmark
| | - J S Walbech
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Lykkebækvej 1, 4600, Køge, Denmark
| | - P Rijnbeek
- Department of Medical Informatics, Erasmus University Medical Centre, Rotterdam, The Netherlands
| | - I Drakos
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Lykkebækvej 1, 4600, Køge, Denmark
| | - I Gögenur
- Center for Surgical Science, Department of Surgery, Zealand University Hospital Køge, Lykkebækvej 1, 4600, Køge, Denmark
| |
Collapse
|
48
|
Yu Y, Peng C, Zhang Z, Shen K, Zhang Y, Xiao J, Xi W, Wang P, Rao J, Jin Z, Wang Z. Machine Learning Methods for Predicting Long-Term Mortality in Patients After Cardiac Surgery. Front Cardiovasc Med 2022; 9:831390. [PMID: 35592400 PMCID: PMC9110683 DOI: 10.3389/fcvm.2022.831390] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Accepted: 03/21/2022] [Indexed: 11/21/2022] Open
Abstract
Objective: This study aims to construct and validate several machine learning (ML) algorithms to predict long-term mortality and identify risk factors in unselected patients post-cardiac surgery. Methods The Medical Information Mart for Intensive Care (MIMIC-III) database was used to perform a retrospective administrative database study. Candidate predictors consisted of the demographics, comorbidity, vital signs, laboratory test results, scoring systems, and treatment information on the first day of ICU admission. Four-year mortality was set as the study outcome. We used the ML methods of logistic regression (LR), artificial neural network (NNET), naïve bayes (NB), gradient boosting machine (GBM), adapting boosting (Ada), random forest (RF), bagged trees (BT), and eXtreme Gradient Boosting (XGB). The prognostic capacity and clinical utility of these ML models were compared using the area under the receiver operating characteristic curves (AUC), calibration curves, and decision curve analysis (DCA). Results Of 7,368 patients in MIMIC-III included in the final cohort, a total of 1,337 (18.15%) patients died during a 4-year follow-up. Among 65 variables extracted from the database, a total of 25 predictors were selected using recursive feature elimination and included in the subsequent analysis. The Ada model performed best among eight models in both discriminatory ability with the highest AUC of 0.801 and goodness of fit (visualized by calibration curve). Moreover, the DCA shows that the net benefit of the RF, Ada, and BT models surpassed that of other ML models for almost all threshold probability values. Additionally, through the Ada technique, we determined that red blood cell distribution width (RDW), blood urea nitrogen (BUN), SAPS II, anion gap (AG), age, urine output, chloride, creatinine, congestive heart failure, and SOFA were the Top 10 predictors in the feature importance rankings. Conclusions The Ada model performs best in predicting 4-year mortality after cardiac surgery among the eight ML models, which might have significant application in the development of early warning systems for patients following operations.
Collapse
Affiliation(s)
- Yue Yu
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Chi Peng
- Department of Health Statistics, Naval Medical University, Shanghai, China
| | - Zhiyuan Zhang
- Department of Cardiothoracic Surgery, No. 988 Hospital of Joint Logistic Support Force of PLA, Zhengzhou, China
| | - Kejia Shen
- Department of Personnel Administration, Second Affiliated Hospital of Naval Medical University, Shanghai, China
| | - Yufeng Zhang
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Jian Xiao
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Wang Xi
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Pei Wang
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Jin Rao
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Zhichao Jin
- Department of Health Statistics, Naval Medical University, Shanghai, China
- *Correspondence: Zhichao Jin
| | - Zhinong Wang
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
- Zhinong Wang
| |
Collapse
|
49
|
Taha-Mehlitz S, Däster S, Bach L, Ochs V, von Flüe M, Steinemann D, Taha A. Modern Machine Learning Practices in Colorectal Surgery: A Scoping Review. J Clin Med 2022; 11:2431. [PMID: 35566555 PMCID: PMC9100508 DOI: 10.3390/jcm11092431] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2022] [Revised: 03/12/2022] [Accepted: 03/29/2022] [Indexed: 12/09/2022] Open
Abstract
OBJECTIVE The use of machine learning (ML) has revolutionized every domain of medicine. Surgeons are now using ML models for disease detection and outcome prediction with high precision. ML-guided colorectal surgeries are more efficient than conventional surgical procedures. The primary aim of this paper is to provide an overview of the latest research on "ML in colorectal surgery", with its viable applications. METHODS PubMed, Google Scholar, Medline, and Cochrane library were searched. RESULTS After screening, 27 articles out of 172 were eventually included. Among all of the reviewed articles, those found to fit the criteria for inclusion had exclusively focused on ML in colorectal surgery, with justified applications. We identified existing applications of ML in colorectal surgery. Additionally, we discuss the benefits, risks, and safety issues. CONCLUSIONS A better, more sustainable, and more efficient method, with useful applications, for ML in surgery is possible if we and data scientists work together to address the drawbacks of the current approach. Potential problems related to patients' perspectives also need to be resolved. The development of accurate technologies alone will not solve the problem of perceived unreliability from the patients' end. Confidence can only be developed within society if more research with precise results is carried out.
Collapse
Affiliation(s)
- Stephanie Taha-Mehlitz
- Clarunis, University Center for Gastrointestinal and Liver Diseases, St. Clara Hospital—University Hospital Basel, 4002 Basel, Switzerland; (S.T.-M.); (S.D.); (M.v.F.); (D.S.)
| | - Silvio Däster
- Clarunis, University Center for Gastrointestinal and Liver Diseases, St. Clara Hospital—University Hospital Basel, 4002 Basel, Switzerland; (S.T.-M.); (S.D.); (M.v.F.); (D.S.)
| | - Laura Bach
- Faculty of Medicine, Friedrich-Alexander University, 91054 Erlangen, Germany;
| | - Vincent Ochs
- Roche Innovation Center Basel, Department of Pharma Research & Early Development, 4070 Basel, Switzerland;
| | - Markus von Flüe
- Clarunis, University Center for Gastrointestinal and Liver Diseases, St. Clara Hospital—University Hospital Basel, 4002 Basel, Switzerland; (S.T.-M.); (S.D.); (M.v.F.); (D.S.)
| | - Daniel Steinemann
- Clarunis, University Center for Gastrointestinal and Liver Diseases, St. Clara Hospital—University Hospital Basel, 4002 Basel, Switzerland; (S.T.-M.); (S.D.); (M.v.F.); (D.S.)
| | - Anas Taha
- Department of Biomedical Engineering, Faculty of Medicine, University of Basel, 4123 Allschwil, Switzerland
| |
Collapse
|
50
|
Dhiman P, Ma J, Andaur Navarro CL, Speich B, Bullock G, Damen JAA, Hooft L, Kirtley S, Riley RD, Van Calster B, Moons KGM, Collins GS. Methodological conduct of prognostic prediction models developed using machine learning in oncology: a systematic review. BMC Med Res Methodol 2022; 22:101. [PMID: 35395724 PMCID: PMC8991704 DOI: 10.1186/s12874-022-01577-x] [Citation(s) in RCA: 50] [Impact Index Per Article: 16.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2021] [Accepted: 03/18/2022] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND Describe and evaluate the methodological conduct of prognostic prediction models developed using machine learning methods in oncology. METHODS We conducted a systematic review in MEDLINE and Embase between 01/01/2019 and 05/09/2019, for studies developing a prognostic prediction model using machine learning methods in oncology. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement, Prediction model Risk Of Bias ASsessment Tool (PROBAST) and CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) to assess the methodological conduct of included publications. Results were summarised by modelling type: regression-, non-regression-based and ensemble machine learning models. RESULTS Sixty-two publications met inclusion criteria developing 152 models across all publications. Forty-two models were regression-based, 71 were non-regression-based and 39 were ensemble models. A median of 647 individuals (IQR: 203 to 4059) and 195 events (IQR: 38 to 1269) were used for model development, and 553 individuals (IQR: 69 to 3069) and 50 events (IQR: 17.5 to 326.5) for model validation. A higher number of events per predictor was used for developing regression-based models (median: 8, IQR: 7.1 to 23.5), compared to alternative machine learning (median: 3.4, IQR: 1.1 to 19.1) and ensemble models (median: 1.7, IQR: 1.1 to 6). Sample size was rarely justified (n = 5/62; 8%). Some or all continuous predictors were categorised before modelling in 24 studies (39%). 46% (n = 24/62) of models reporting predictor selection before modelling used univariable analyses, and common method across all modelling types. Ten out of 24 models for time-to-event outcomes accounted for censoring (42%). A split sample approach was the most popular method for internal validation (n = 25/62, 40%). Calibration was reported in 11 studies. Less than half of models were reported or made available. CONCLUSIONS The methodological conduct of machine learning based clinical prediction models is poor. Guidance is urgently needed, with increased awareness and education of minimum prediction modelling standards. Particular focus is needed on sample size estimation, development and validation analysis methods, and ensuring the model is available for independent validation, to improve quality of machine learning based clinical prediction models.
Collapse
Affiliation(s)
- Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK.
- NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
| | - Jie Ma
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK
| | - Constanza L Andaur Navarro
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Benjamin Speich
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK
- Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Garrett Bullock
- Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Johanna A A Damen
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Lotty Hooft
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Shona Kirtley
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK
| | - Richard D Riley
- Centre for Prognosis Research, School of Medicine, Keele University, Staffordshire, ST5 5BG, UK
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands
- EPI-centre, KU Leuven, Leuven, Belgium
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK
- NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| |
Collapse
|