Prediction of permanent pacemaker implantation after transcatheter aortic valve replacement: The role of machine learning

Abstract

BACKGROUND

Atrioventricular block requiring permanent pacemaker (PPM) implantation is an important complication of transcatheter aortic valve replacement (TAVR). Application of machine learning could potentially be used to predict pre-procedural risk for PPM.

AIM

To apply machine learning to be used to predict pre-procedural risk for PPM.

METHODS

A retrospective study of 1200 patients who underwent TAVR (January 2014-December 2017) was performed. 964 patients without prior PPM were included for a 30-d analysis and 657 patients without PPM requirement through 30 d were included for a 1-year analysis. After the exclusion of variables with near-zero variance or ≥ 50% missing data, 167 variables were included in the random forest gradient boosting algorithm (GBM) optimized using 5-fold cross-validations repeated 10 times. The receiver operator curve (ROC) for the GBM model and PPM risk score models were calculated to predict the risk of PPM at 30 d and 1 year.

RESULTS

Of 964 patients included in the 30-d analysis without prior PPM, 19.6% required PPM post-TAVR. The mean age of patients was 80.9 ± 8.7 years. 42.1 % were female. Of 657 patients included in the 1-year analysis, the mean age of the patients was 80.7 ± 8.2. Of those, 42.6% of patients were female and 26.7% required PPM at 1-year post-TAVR. The area under ROC to predict 30-d and 1-year risk of PPM for the GBM model (0.66 and 0.72) was superior to that of the PPM risk score (0.55 and 0.54) with a P value < 0.001.

CONCLUSION

The GBM model has good discrimination and calibration in identifying patients at high risk of PPM post-TAVR.

Key Words: Transcatheter aortic valve replacement, Permanent pacemaker implantation, Machine learning

Core Tip: Atrioventricular block requiring permanent pacemaker (PPM) implantation is an important complication of transcatheter aortic valve replacement. Application of machine learning could potentially be used to predict pre-procedural risk for PPM. Machine learning was used to predict patients who are at risk of developing conduction abnormalities requiring PPM at 30 d and 1 year. Our random forest machine learning model using machine learning outperforms PPM risk score model in its predictive value. Brachiocephalic to annulus distance to height ratio is the highest weighted predictor of PPM implantation at both 30-d and 1-year, which has not been previously described in the literature.

INTRODUCTION

Transcatheter aortic valve replacement (TAVR) is increasingly being used in preference to surgical aortic valve replacement (SAVR) in patients with aortic stenosis[1-3]. The most common complication of TAVR remains the development of atrioventricular conduction abnormalities, requiring permanent pacemaker (PPM) implantation, despite the use of improved implant performance and newer generation valves[4-12]. PPM is associated with increased length of hospital stay and mortality[13]. Additionally, advanced conduction defects requiring PPM implantation have been demonstrated to lead to worse functional capacity and clinical outcomes in patients with aortic stenosis[1-4]. The PPM requirement rate in TAVR is two to five-fold higher than in SAVR[15,16]. Certain baseline characteristics such as age, gender, pre-existing atrioventricular block, right bundle branch block, left bundle branch block[17,18], and size of the left ventricular outflow tract (LVOT), as well as procedure-related factors such as implantation depth have been shown to be associated with PPM requirement risk. Previous studies that evaluated risk factors associated with PPM requirement used data for older-generation valves and included only a limited number of variables, thus limiting their predictive potential[11,13,19,20]. Consequently, it is very important to risk stratify patients for potential need of PPM implantation post-procedure. Artificial intelligence (AI) refers broadly to analytical algorithms that iteratively learn from data, enabling machines to find hidden insights without the need for explicit programming where to look[21-24]. Machine learning (ML) is a computer science sector that uses computer algorithms to identify patterns with a multitude of variables in large datasets and thereby anticipates various data-based outcomes[25]. In this study, we used supervised ML with the gradient boosting machine learning model (GBM) to predict pre-procedural risk for PPM post-TAVR at 30 d and 1 year.

MATERIALS AND METHODS

We performed a retrospective study on all patients with severe symptomatic aortic stenosis who underwent TAVR at the Mayo Clinic hospitals in Rochester, MN, Phoenix, AZ, and Jacksonville, FL between January 1, 2012, and December 30, 2017. The Mayo Clinic Institutional Review Board (IRB) approved the study protocol and research authorization to utilize medical information for clinical research was provided by the patients. A retrospective chart review of the electronic health record was used to collect baseline data, and clinical coordinators were contacted for information on follow-up visits. We identified 285 clinical variables for potential inclusion into the ML algorithm.

Out of 1200 patients, 236 individuals with prior pacemakers were excluded. The remaining 964 patients were included in the 30-d PPM risk prediction analysis. We first eliminated all variables with ≥ 50% missing and near-zero variance, where variables with near-zero variance have one unique value or the majority of the data is comprised within a single category. The GBM algorithm handles missing data internally by treating “missing” as its own category. This left 147 out of 285 variables to be included in the model. These variables were used to predict the risk of pacemakers 30 d post-TAVR using the GBM model. The model was optimized using 5-fold cross-validation repeated 10 times to get the highest prediction accuracy. Among the 964 patients without prior PPM who have undergone TAVR, 189 patients required PPM implantation by 30 d, 116 patients were deceased by 1 year, and 2 patients were lost to follow-up, leaving 657 patients who were included in the final analysis to predict the need for PPM at 1 year. There were 287 variables initially, but all variables with ≥50% missing or near-zero variance were eliminated leaving a total of 163 variables. Patient recruitment is summarized in Figure 1.

Figure 1 Flowchart depicting patient recruitment for the analysis transcatheter aortic valve replacement-transcatheter aortic valve replacement. TAVR: Transcatheter aortic valve replacement.

Clinical variables, comorbidities, and procedural factors were obtained from chart review. Definitions conformed to those provided by the Transcatheter Valve Therapy (TVT) Registry[26]. Echocardiographic variables were collected using standard ultrasound scanners. Comprehensive Doppler and 2-Dimensional Transthoracic Echocardiogram (TTE) were performed prior to the procedure. TTE images were acquired and interpreted according to the European Association of Echocardiography and American Society of Echocardiography guidelines. Multi-detector computed tomography (MDCT) was performed a month before the treatment. The size of the aortic annulus was determined pre-procedure.

Statistical analysis

The study population data set (n = 964 and n = 657) for 30 d and 1 year, respectively, had low event rates. Due to a small percentage of events, the entire data set was used in the modeling phase and was not broken into a test and train cohort. The caret R package was used to fit a GBM model from the gbm³R package using 5-fold cross-validation repeated 10 times. Model hyperparameters, specified prior to fitting the model, are tunable variables that control the chosen model’s learning process. The hyperparameters tuned were the interaction depth, number of trees, and shrinkage. The minimum number of observations required at each node was fixed at 20. Figures 2 and 4 include the top 20 variables that indicate which have the highest predictive power in classifying those with events and those without events. The study population for PPM risk was limited to those that had a trans-femoral or trans-apical approach. The PPM risk score developed by Vejpongsa et al[20] uses 6 factors. Each factor had points associated that collapsed into a three-group score (low, moderate, or high risk). Tuning of hyperparameters optimizes the target metric, that metric being the area under the receiver operating characteristic curve (AUC). AUC is a numeric metric that measures how well the model can distinguish between patients with PPM and those without PPM.

Figure 2 Variables with the highest importance in a gradient boosting model to predict the need for a permanent pacemaker at 30 d.

Figure 3 Receiver operator curves of the gradient boosting model and permanent pacemaker risk score model to predict the need for a permanent pacemaker at 30 d. GBM: Gradient boosting model; PPM: Permanent pacemaker model.

Figure 4 Variables with the highest importance in the gradient boosting model to predict the need for a permanent pacemaker at 1 year.

The predicted probabilities that were generated on each fold were stacked, which was repeated 10 times for each patient. The model took the average of the predicted probabilities of all 10 repeats; the average predicted probabilities for each patient were then used to compute the final AUC. The pROC R package was used to produce the ROC curves along with the 95%CI for the AUC (Figures 3 and 5). Variable importance is determined by calculating the relative influence of each variable included in the model. The variable importance plot provides a ranked list of the most significant variables in descending order.

Figure 5 Receiver operator curves curves of gradient boosting model and permanent pacemaker risk score model predicting the need for a permanent pacemaker at 1 year. GBM: Gradient boosting model; PPM: Permanent pacemaker model.

The caret R package was used to fit a logistic regression using 5-fold cross-validation repeated 10 times. Similar to the GBM model, this process also used 5-fold cross-validation repeated 10 times, where the predicted probabilities for each fold were stacked and then averaged over all 10 repeats for each patient. The average predicted probabilities of PPM risk for each patient were used to produce the final AUC. Categorical and ordinal variables were compared either with the chi-square or Fisher exact tests and are expressed as numbers and percentages. Continuous variables were compared with the t-test and expressed as mean ± SD. Pearson’s χ² test and Analysis of Variance were used to assess the baseline differences. A P < 0.05 was considered significant. R software version 3.4.1 (Foundation of Statistical Computing, Vienna, Austria) was used to run the analysis. Baseline characteristics, echocardiographic variables, EKG variables, and MDCT variables for 30 d and 1-year analysis are shown in the Supplementary material. Marlene Girardo and Matthew Buras are the statisticians who ran the analysis and are also authors of the paper.

RESULTS

30-d analysis

The mean age of the patients was 80.9 ± 8.7. 42.1% of patients were female and 19.6% (n = 189) required PPM at 30 d post-TAVR. 68.8% of the entire patient cohort had a balloon-expandable valve. Patients requiring PPM post-TAVR had higher proportions of prior percutaneous coronary interventions, aspirin use, trans-femoral access, self-expandable valve use, and New York Heart Association heart failure class III/IV as compared to those who did not require PPM post-TAVR. Other baseline differences between the two groups can be seen in the Supplementary material. Using our GBM machine learning algorithm, a scoring model using the 20 highest weighted predictors of PPM requirement post-TAVR was generated. The highest weighted characteristic was a higher brachiocephalic artery to annulus distance to patient height ratio, followed by right bundle branch block (RBBB), higher brachiocephalic to aortic annulus distance, high pre-operative risk, and the use of self-expanding valves (as opposed to balloon expandable valves). Figure 2 shows the full list with the relative weights of the twenty variables. The area under ROC to predict the need for PPM at 30 d for the GBM model was 0.66 (95%CI: 0.61-0.70) vs 0.55 (95%CI: 0.49-0.60) for the PPM risk score model (P < 0.001). The comparison of the ROC curves of both models is shown in Figure 3.

1-year analysis

The mean age of the patients was 80.7 ± 8.2. 42.6% of patients were female and 26.7% (n = 176) required PPM at 1-year post-TAVR. 67.6% of the entire patient cohort had a balloon-expandable valve. Patients requiring PPM at 1-year post-TAVR had higher proportions of prior aortic valve intervention, aspirin use, severe mitral stenosis, elevated filling pressures, and percutaneous transfemoral access compared to those who did not require PPM at 1 year. Other baseline differences can be seen in the Supplementary material. Based on the GBM machine learning algorithm, a scoring model using the 20 highest weighted predictors of PPM dependency at 1-year post-TAVR was generated. The five highest weighted predictors were higher brachiocephalic artery to annulus distance to height ratio, higher mitral valve diastolic mean gradient, RBBB, higher LVOT diameter, and higher distance of right coronary artery to basal ring (mm). Figure 4 shows all twenty variables with the highest weightage. The area under ROC to predict the need for PPM at 1 year for the GBM model was 0.72 (95%CI: 0.67-0.76) vs 0.54 (95%CI: 0.49-0.60) for the PPM risk score model (P value < 0.001). The comparison of the ROC curves of both models is shown in Figure 5.

DISCUSSION

Given the clinical relevance of conduction abnormalities necessitating PPM, we sought to develop a risk assessment tool to predict PPM implantation in patients post-TAVR using machine learning (ML). ML seeks to mimic the thought process, learning capacity, and storage of knowledge of humans[28]. Its techniques have been in use in cardiovascular medicine, but our study is the first to predict the risk of PPM implantation in patients post-TAVR. This study demonstrates that ML could be used to accurately predict the requirement of PPM at 1-year post-TAVR with a high level of discriminatory ability. The GBM model had a modest level of discriminatory ability to predict the requirement of PPM at 30 d. Arteriovenous conduction disturbances are well-known post-TAVR. The most common conduction abnormalities post-TAVR are left bundle branch block (LBBB) and complete heart block[30,31]. Multiple mechanistic reasons for these abnormalities have been theorized, and the most popular one is that the spatial proximity of the cardiac conduction system to the calcified aortic valve[32,33], as well as the underlying conduction disease prevalence in this elderly group[34], predisposes it to damage during the TAVR procedure. Many patients require placement of PPM post-TAVR, with an incidence of 10%-15% commonly cited in the literature, with substantial variability based on the specific TAVR valve used[4]. Conduction abnormalities are clinically relevant as these patients have a higher incidence of subsequent hospitalizations, less improvement in LV function and functional status after TAVR, and possibly even higher mortality, though there is conflicting evidence regarding the latter and long-term prognosis[11,13,30,35].

The rate of PPM implantation post-TAVR in our study was 19.6% at 30 d and 26.7% at 1 year, which is similar to previous trials[8,36-39]. Pre-existing conduction abnormalities such as RBBB, LBBB, and 1st-degree AV block were significantly associated with post-TAVR PPM implantation, and these are consistent with the previous studies[12,13]. Trans-femoral access was also significantly correlated with the PPM rate, which has also been described as a risk factor in a prior registry[13]. Another variable that strongly associates with the PPM rate was self-expanding valves which are also known through prior studies[12,13]. High rates (13.3%-17%) of implantation with the Edwards Sapien 3 valve have previously been demonstrated which was also consistent with our study[19,36,40]. Brachiocephalic artery to aortic valve annulus distance to height ratio was the highest weighted predictor for PPM implantation post-TAVR at both one month and one year. As far as we are aware, we are the first to describe this variable as a predictor for PPM requirement, let alone as the highest weight predictor. It is not clear why it is associated with conduction abnormalities requiring PPM. We suspect that the longer distance of the ascending aorta proximal to the origin of the brachiocephalic artery allows for the TAVR valve to hug the outer curve of the aorta more, thus exerting more force on the right/non-cusp side where the conduction system lies. This needs to be confirmed in other studies.

Overall, the model used for the 30-d and 1-year predictors yielded a very similar set of variables. The main difference was the presence of mitral valve diastolic mean gradient on echo which was the second highest weighted predictor for PPM at 1 year but was not present in the 30-d predictive model. Whether it is the gradient itself that is associated with conduction abnormalities or the mitral annular calcification that is presumably associated with such gradients and would be expected in such populations with calcific aortic stenosis is unclear. The mitral valve and annular calcification were not one of our echocardiographic parameters that were included in the study, so further studies need to be completed. The subsequent evaluation of whether mitral valve or annular calcification is associated with conduction abnormalities independent of AS and TAVR is an obvious corollary. The comparison of our predictive model with the PPM risk score developed by Vejpongsa et al[20] which uses 6 variables for scoring, demonstrates the enhanced prognostic capability of our model (Figures 3 and 5). Other risk score models for PPM requirement post-TAVR that have been described are the Emory Risk Score developed by Kiani et al[19] and the risk score developed by Maeno et al[41]. We were unable to compare our model with these risk score models due to a lack of complete variables, including the history of syncope in the Emory risk score, and membranous septum (MS) length in the risk score. Some of the limitations of this study need to be noted. Firstly, the model is complex, and therefore its use may be limited in clinical practice. Additionally, given the large number of demographic information and clinical variables included in this model, these variables may not always be present. Nevertheless, we feel that the prognosticating ability of the model overcomes this limitation and that with the increasing use of electronic medical records, most data is available. Secondly, this was primarily a feasibility study and is retrospective in nature, which restricts our ability for defining causal associations. There is a need for prospective validation with an external cohort. Thirdly, we did not include a few variables in our model that have been included in other risk scores for PPM implantation, such as a history of syncope or distal landing zone calcium burden, as these variables were not present in enough of our cohort to include. Thus, there is a potential for change in the analysis with the inclusion of such variables. Lastly, the study included primarily referred patients in three high-volume tertiary care centers, and thus are likely higher risk and more complex than the average TAVR patient.

CONCLUSION

Machine learning was used to predict patients who are at risk of developing conduction abnormalities requiring PPM at 30 d and 1 year. Our GBM model using machine learning outperforms the PPM risk score model in its predictive value. Brachiocephalic to annulus distance to height ratio is the highest weighted predictor of PPM implantation at both 30 d and 1 year, which has not been previously described in the literature.

ARTICLE HIGHLIGHTS

Research background

For aortic stenosis, it is a fact that transcatheter aortic valve replacement use has greatly increased relative to surgical replacement with the most common complications of the procedure including atrioventricular conduction abnormalities development and permanent pacemaker requirement (PPM). Hence, it is essential to risk stratify patients for potential need of PPM implantation post-procedure. We used artificial intelligence to predict pre-procedural risk for pacemaker placement post-transcatheter aortic valve replacement at 30 d and 1 year.

Research motivation

Previous studies that evaluated risk factors associated with permanent pacemaker requirement used data for older-generation valves and also included only a limited number of variables and hence, limiting their predictive potential. Artificial intelligence does a remarkable job of predicting variables via machine learning and the same has been used in our study.

Research objectives

To predict pre-procedural risk for permanent pacemaker post-transcatheter aortic valve replacement (TAVR) at 30 d and 1 year.

Research methods

We performed a retrospective study on patients with severe symptomatic aortic stenosis who underwent transcatheter aortic valve replacement (TAVR). Gradient boosting machine learning model has been used for predicting probabilities.

Research results

For 30-d analysis, higher brachiocephalic artery to annulus distance to patient height ratio was the highest weighted characteristic that predicted PPM placement post- TAVR. Also for 1-year analysis, higher brachiocephalic artery to annulus distance to patient height ratio was the highest weighted characteristic that predicted PPM placement post- TAVR.

Research conclusions

Brachiocephalic to annulus distance to height ratio is the highest weighted predictor of PPM implantation in the study both at 30 d and 1 year and it was not been previously described in the literature.

Research perspectives

We sought to develop and have developed a risk assessment tool to predict PPM implantation post-TAVR using machine learning.

ACKNOWLEDGEMENTS

This publication was funded by Mayo Clinic Arizona Cardiovascular Clinical Research Center (MCA CV CRC). We are thankful for their generous support.