This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Author contributions: All authors equally contributed to data collection, interpretation, and manuscript writing.
Conflict-of-interest statement: The authors have nothing to disclose.
PRISMA 2009 Checklist statement: The authors have read the PRISMA 2009 Checklist, and the manuscript was prepared and revised according to the PRISMA 2009 Checklist.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Nathan J Smischney, MD, MSc, Assistant Professor, Department of Anesthesia, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States. email@example.com
Received: August 1, 2020 Peer-review started: August 1, 2020 First decision: September 17, 2020 Revised: October 10, 2020 Accepted: October 26, 2020 Article in press: October 26, 2020 Published online: December 18, 2020
Vasoplegic shock is a challenging complication of cardiac surgery and is often resistant to conventional therapies for shock. Norepinephrine and epinephrine are standards of care for vasoplegic shock, but vasopressin has increasingly been used as a primary pressor in vasoplegic shock because of its unique pharmacology and lack of inotropic activity. It remains unclear whether vasopressin has distinct benefits over standard of care for patients with vasoplegic shock.
To summarize the available literature evaluating vasopressin vs non-vasopressin alternatives on the clinical and patient-centered outcomes of vasoplegic shock in adult intensive care unit (ICU) patients.
This was a systematic review of vasopressin in adults (≥ 18 years) with vasoplegic shock after cardiac surgery. Randomized controlled trials, prospective cohorts, and retrospective cohorts comparing vasopressin to norepinephrine, epinephrine, methylene blue, hydroxocobalamin, or other pressors were included. The primary outcomes of interest were 30-d mortality, atrial/ventricular arrhythmias, stroke, ICU length of stay, duration of vasopressor therapy, incidence of acute kidney injury stage II-III, and mechanical ventilation for greater than 48 h.
A total of 1161 studies were screened for inclusion with 3 meeting inclusion criteria with a total of 708 patients. Two studies were randomized controlled trials and one was a retrospective cohort study. Primary outcomes of 30-d mortality, stroke, ventricular arrhythmias, and duration of mechanical ventilation were similar between groups. Conflicting results were observed for acute kidney injury stage II-III, atrial arrhythmias, duration of vasopressors, and ICU length of stay with higher certainty of evidence in favor of vasopressin serving a protective role for these outcomes.
Vasopressin was not found to be superior to alternative pressor therapy for any of the included outcomes. Results are limited by mixed methodologies, small overall sample size, and heterogenous populations.
Core Tip: In this systematic review of vasopressin vs alternative vasoactive agents for the treatment of vasoplegic shock, vasopressin was not found to be superior to alternative pressor therapy for any of the included outcomes. However, results are limited by mixed methodologies, small overall sample size, and heterogenous populations.
Citation: Webb AJ, Seisa MO, Nayfeh T, Wieruszewski PM, Nei SD, Smischney NJ. Vasopressin in vasoplegic shock: A systematic review. World J Crit Care Med 2020; 9(5): 88-98
Vasoplegic shock, one of the most significant complications that can arise after cardiac surgery, can be devastating and challenging to manage. Vasoplegic shock is defined by low systemic vascular resistance despite adequate fluid resuscitation and a normal or increased cardiac index. Post-operative vasoplegia is most common after cardiac surgery involving cardiopulmonary bypass, occurring in about 5% to 25% of patients. While vasoplegic shock can occur after non-cardiac surgery, the most common risk factors for vasoplegia include cardiopulmonary bypass and the use of angiotensin converting enzyme inhibitors and beta blockers prior to surgery[1,5].
Vasoplegic shock involves both hyperactivity of vasodilatory pathways and resistance to and deficiency of common vasoconstrictor pathways[6,7]. Patients have been observed to mount a profound inflammatory response to cardiopulmonary bypass, leading to increased expression of nitric oxide synthase, decreased levels of vasopressin, and altered activity of catecholamine-sensitive secondary messenger systems[8,9]. Catecholamines, especially norepinephrine, have long been considered first line, but evidence supporting one therapy over another is limited and each carry the risk of adverse effects[10,11]. Other therapeutic agents targeting different pathophysiologic complications of vasoplegia include methylene blue, hydroxo-cobalamin, vasopressin, and angiotensin II and each carries distinct potential benefits and risks.
Vasopressin’s unique pharmacology may lend it to being particularly beneficial in vasoplegic shock[12-15]. Activation of Gq-coupled vasopressin-1 (V1) receptors leads to smooth muscle contraction through the recruitment of intracellular calcium stores in the sarcoplasmic reticulum and extracellular calcium stores by opening L-type calcium channels[16,17]. There is also minimal V1 receptor expression in the pulmonary vasculature which may be of particular benefit to patients with right heart dysfunction or pulmonary hypertension. Questions still remain, however, about its benefits over standard of care in shock. There is a lack of large, multi-center prospective trials addressing these questions. Thus, the aim of this systematic review was to summarize the available literature evaluating vasopressin vs non-vasopressin alternatives on the clinical and patient-centered outcomes of vasoplegic shock in adult intensive care unit (ICU) patients.
MATERIALS AND METHODS
This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) 2015 guidelines. A formal protocol does not exist for this systematic review.
We included randomized controlled trials, prospective cohort studies, and retrospective cohort studies published in English in peer-reviewed journals. Studies were included if they studied adult patients (≥ 18 years), compared vasopressin to norepinephrine, epinephrine, hydroxocobalamin, or methylene blue, evaluated patients treated in the intensive care unit, and were suffering from post-operative vasoplegic shock. Follow-up needed to be until at least 30 d post-discharge. Studies needed to report 30-d mortality, acute kidney injury stage II-III based on Acute Kidney Injury Network classification (reference), safety, ICU length of stay, mechanical ventilation duration, and duration of vasopressor therapy We excluded studies in pediatric patients, case reports, case series, review articles, letters, and notes. No restrictions were placed on the location of publication.
A comprehensive search of several databases from each database's inception to December 6, 2019 of any language was conducted. The databases included Ovid MEDLINE(R) and Epub Ahead of Print, In-Process and Other Non-Indexed Citations, and Daily, Ovid EMBASE, Ovid Cochrane Central Register of Controlled Trials, Ovid Cochrane Database of Systematic Reviews, and Scopus. The search strategy was designed and conducted by an experienced librarian with input from the reviewers. Controlled vocabulary supplemented with keywords was used to search for studies of vasoplegia/vasoplegic shock in critically ill patients. Actual strategy listing all search terms used and how they were combined is available in Supplementary 1.
Article titles and abstracts were screened by two independent authors (MOS and TN) for inclusion based on the aforementioned inclusion and exclusion criteria. The full text of articles included by title and abstract were then reviewed and disagreements were resolved through consensus.
The primary outcomes of interest were 30-d mortality, atrial/ventricular arrhythmias, vasopressor duration, stroke, ICU length of stay, proportion of patients suffering acute kidney injury, defined as acute kidney injury network stage 2 (serum creatinine [SCr] increase of 200% or urine output less than 0.5 mL/kg per hour in a 12 h period) or 3 (SCr increase of 300% or SCr greater than or equal to 4 mg/dL with an acute rise of at least 0.5 mg/dL or a urine output of less than 0.3 mL/kg/h in a 24 h period or anuria for 12 h), and proportion of patients mechanically ventilated for greater than 48 h.
Methodological quality and certainty of evidence
The Cochrane Collaboration tool for assessing risk of bias was utilized to assess the quality and bias risk of included randomized controlled studies. The tool assesses studies based on randomization, protocol deviation, missing outcome data, outcome measurements, and result reporting. The Newcastle Ottawa scale was used for assessing the risk of bias in observational studies. The tool assesses studies based on selection methods, comparability, and outcome measurements. Discrepancies in scoring were resolved through consensus.
Two independent authors (MOS and TN) reviewed and extracted relevant data from included manuscripts in a standard data collection form. Collected data included publication information, protocol details, outcome measures, baseline characteristics, and results.
For continuous outcomes, we gathered means and variance data [e.g., standard deviation, standard error, confidence interval (CI)] and the weighted mean difference (MD). For binary outcomes, we gathered incidence data and frequencies and calculated the relative risk (RR). All statistical analyses were performed using R Core Team version 4.0.0 (2020).
The initial search identified 1161 studies. Following removal of duplicates and excluded records, 115 full-text articles were assessed for eligibility. Three (2.6%) of these met the inclusion criteria and were included in the analysis[23-25]. The results of the systematic search are summarized in Figure 1.
Figure 1 Study flow diagram.
Of the 3 included studies, 2 were randomized controlled trials[24,25] and 1 was a retrospective cohort study. A total of 1496 participants were included across the 3 studies (Table 1). The included studies were performed in Egypt, China, and Brazil, and publication dates spanned from 2016 to 2018. Characteristics of all of the included studies are detailed in Table 1.
Severe sepsis diagnosed within 72 h and septic shock diagnosed within 24 h from the time of giving norepinephrine dose of greater than or equal to 0.2 µg/kg per minute, which is required to maintain the mean arterial pressure between 70 and 90 mmHg
(1) Pregnant females; (2) Patients sensitive to Methylene blue or vasopressin; (3) Patients with known G6PD deficiency; (4) Age less than 18 yr; (5) Vasospastic diathesis (e.g., Raynaud’s syndrome); (6) Coronary artery disease; and (7) Patients receiving mono amine oxidase inhibitors
Methylene blue (20); vasopressin (20)
55.3 ± 20.9; 59.4 ± 14.5
ICU length of stay; mean arterial pressure; central venous pressure; pulmonary artery pressure
Patients with age more than 18 yr, who had left ventricular ejection fraction ≤ 35%, left ventricular end-diastolic diameter ≥ 60 mm, and New York Heart Association ≥ III), and developing postoperative vasoplegic shock (mean arterial pressure < 65 mmHg resistant to fluid challenge and cardiac index > 2.20 L/min per meter squared)
(1) Patients with chronic obstructive pulmonary disease; and (2) Adult congenital heart disease
All adult (more than 18 yr of age) patients who were scheduled for coronary artery bypass graft surgery, valve replacement, or repair surgery with cardiopulmonary bypass who required vasopressor drugs for vasodilatory shock within 48 h after coronary artery bypass surgery weaning
(1) Aortic surgery; (2) Heart transplantation; (3) Preoperative use of vasopressor therapy; (4) Presence of a ventricular assist device other than an intra-aortic balloon pump; (5) Severe hyponatremia (< 130 mEq/L); (6) Acute coronary syndrome; (7) Acute mesenteric ischemia; (8) History of Raynaud disease; (9) Pregnancy; and (10) Neoplasm
Norepinephrine (151); vasopressin (149)
55 ± 13; 54 ± 14
Days alive and free of organ dysfunction at 28 d; stroke; acute renal failure; 30 d incidence of infection, septic shock, arrhythmias (atrial fibrillation and ventricular arrhythmias); duration of mechanical ventilation; changes in hemodynamic variables; the use of dobutamine or other vasoactive agents); incidence of digital ischemia; acute mesenteric ischemia; acute myocardial; infarction; ICU and hospital lengths of stay
G6PD: Glucose-6-phosphate dehydrogenase; ICU: Intensive care unit; mEq/L: Milliequivalents per liter.
Risk of bias
Overall, the risk of bias of the 2 included trials was moderate due to having some concerns in the randomization process of the 2 clinical trials[24,25]. The risk of bias for the cohort study was low. The risk of summary bias is provided in Tables 2 and 3.
Table 2 Risk of summary bias (randomized controlled trials).
Data is presented as effect size, 95% confidence interval (CI), certainty of evidence. CI: Confidence interval; MD: Mean difference; RR: Relative risk.
Thirty days mortality: Two studies were identified which reported 30-d mortality (n = 668)[23,24]. The risk of 30-d mortality was not found to differ between vasopressin as compared with norepinephrine.
Atrial/ventricular arrhythmias and stroke: Only two of the included studies reported safety events (n = 668)[23,24]. Although arrhythmias including atrial fibrillation and ventricular tachycardia occurred at a significantly higher frequency with vasopressin than norepinephrine as reported by Cheng et al the certainty of evidence was low due to study design and imprecision. Hajjar et al reported a similar frequency of ventricular tachycardia between the two pressors and vasopressin demonstrated a favorable profile at reducing atrial fibrillation when compared to norepinephrine. The certainty of evidence in these results was moderate. Although, neither study reported maximum dosage of study drug infusion rate, or dosage of vasopressors at the time of arrhythmia. Both studies did not report any differences in stroke.
Duration of vasopressors: Two studies reported duration of vasopressors (n = 668) [23,24]. The studies report discordant effect with one favoring use of vasopressin (MD -23, 95%CI -36.12, -9.88; moderate certainty of evidence, Hajjar et al), while the other favoring use of norepinephrine (MD 24, 95%CI 16.32, 31.68; very low certainty of evidence, Cheng et al).
ICU length of stay: All three studies reported ICU length of stay, although one study utilized methylene blue as the comparator (n = 40), whereas the other two utilized norepinephrine (n = 668)[23,24]. No differences between vasopressin and methylene blue were found. When vasopressin was compared to norepinephrine, the two studies reported contradictory results with a longer length of stay in Cheng et al (low certainty of evidence) and a shorter length of stay in Hajjar et al (moderate certainty of evidence).
Acute kidney injury: Two studies reported incidence of acute kidney injury stage 2 or 3 (n = 668)[23,24]. Cheng et al reported that vasopressin did not significantly affect the risk of acute kidney injury (very low certainty of evidence) while Hajjar et al demonstrated a considerable reduction in the risk of acute kidney injury when compared to norepinephrine (moderate certainty of evidence). Not enough data in the studies were available to assess need for or eventual dialysis dependency.
Mechanical ventilation > 48 h: Two studies reported outcome data on mechanical ventilation > 48 h (n = 668)[23,24]. Although not significant, vasopressin was associated with less episodes of mechanical ventilation lasting more than 48 h.
In this systematic review of the literature evaluating the role of vasopressin in the treatment of post-operative vasoplegic shock, studies evaluating the effects on 30-d mortality, acute kidney injury stage 2-3, ICU length of stay, atrial fibrillation, ventricular arrhythmias, mechanical ventilation duration, and stroke were summarized. Meta-analysis was not feasible due to differences in methodology, patients, and procedures that led to variation in the reported results between studies.
Interest in vasopressin as treatment for vasoplegic shock has existed for a number of years due to its unique pharmacology independent of the autonomic nervous system. Current available literature, however, has been limited by small sample sizes, inconsistent populations, and varied outcomes, which has limited its use to adjunctive therapy. Insights from investigation into vasopressin’s role in the treatment of septic shock, however, may supplement knowledge on vasopressin’s role in vasoplegic shock. Randomized controlled trials of vasopressin in septic shock have not revealed a significant mortality benefit, but signals of preserved renal function, decreased overall pressor requirements, and largely equitable safety outcomes has changed it from salvage therapy to standard care for many patients with septic shock[26-30].
The evolution of vasopressin in septic shock may foreshadow the role of vasopressin in vasoplegic shock. Norepinephrine and epinephrine have functioned as the workhorses of vasoplegic shock management for decades and clinical experience outweighs the influence of the available literature to support the role of vasopressin. As clinical experience with vasopressin grows alongside the expansion of the literature, vasopressin utilization in vasoplegic shock without cardiogenic shock will likely increase. The results of this systematic review did not reveal any major advantages to vasopressin use but highlight the need for robust investigation into many of these outcomes.
Like other studies investigating specific pressors, 30-d mortality was not found to be different between patients who received vasopressin or norepinephrine in our systematic review. This is concordant with studies evaluating pressors in other shock states as well as studies evaluating vasopressin in septic shock. Few large randomized controlled trials have succeeded in demonstrating a reduction in mortality of a singular critical care intervention, and the benefit of each individual intervention, such as the choice of vasopressor, may be better judged by its incremental benefits on morbidity and patient-specific outcomes[31-33].
No difference was revealed in ICU length of stay for vasopressin compared to norepinephrine or methylene blue in our systematic review. Of note, opposing results were reported in Hajjar et al and Cheng et al This imbalance may in part be due to the different baseline populations in each study, with Hajjar et al excluding patients with left ventricular dysfunction and Cheng et al specifically including these patients, as well as the study design (randomized clinical trial vs cohort study). In a meta-analysis of vasopressin in septic shock, vasopressin has not been reported to have a significant impact on ICU length of stay (mean different -0.08 d, 95%CI -0.68, 0.52).
Vasopressin was not found to impact rates of stroke in patients with vasoplegic shock. Perioperative stroke after cardiac surgery is uncommon, estimated to occur in about 2% of all patients after surgery, but rates of mortality after perioperative stroke are much higher than the overall population[35,36]. While our findings indicate choice of pressor did not influence this risk, the overall sample size may be too low to estimate the impact on a rate outcome (combined event rate was 17). Potential confounders for risk of stroke, such as previous stroke, were not reported.
Given its lack of autonomic activity, one potential benefit of vasopressin is its presumed lack of arrhythmogenic properties. In our analysis, we found conflicting results from the two studies which reported ventricular and atrial arrhythmias as an outcome. This finding contrasts that of a patient-level meta-analysis of adverse event data in septic shock, which found vasopressin was associated with an absolute risk reduction of 2.8% (95%CI -0.2, -5.3) in rates of arrhythmia compared to norepinephrine. Vasopressin with a catecholamine was also found to confer a lower risk of atrial arrhythmia compared to catecholamines alone in a meta-analysis of multiple shock states (RR 0.77, 95%CI 0.67, 0.88). The different results of each study in our systematic review are potentially driven by the unreported doses of pressors in Cheng et al at the time of ventricular arrhythmia onset and the higher vasopressor needs overall in the six hours after cardiac surgery in the vasopressin group, which would be an unaccounted confounder. Of note, one should be aware that the randomized clinical trial, Hajjar et al, demonstrated reduced arrhythmogenic potential for both atrial and ventricular arrhythmias with vasopressin compared to norepinephrine unlike the cohort study of Cheng et al.
The two studies reporting vasopressor duration also had opposite effects. This discrepancy is likely due to differences in methodology and patient populations between the two studies. Considering the heterogeneity between these two studies (see Supplementary 2) and the overall higher level of evidence in Hajjar et al, the beneficial effect on vasopressor duration in Hajjar et al is likely a better representation of the true effect of vasopressin on this outcome, as we demonstrate for the arrhythmia and renal endpoints. Duration of vasopressor therapy may be better reported as days alive and free of vasopressors, a more patient-centered outcome.
Rates of stage II or III acute kidney injury were not found to be different depending on which pressor was used for vasoplegic shock. Vasopressin has unique activity at the glomerulus, including an ability to selectively constrict the efferent arteriole and not the afferent arteriole, leading to an observed increase in urine output in patients with septic shock[14,39]. In a meta-analysis of multiple shock states, vasopressin was revealed to be protective for acute kidney injury compared to alternative therapy (OR 0.52, 95%CI 0.32, 0.86). This analysis, however, is limited by mixing definitions of acute kidney injury, study designs, and indications. Need for renal replacement therapy was also not protocolized and up to the decision of the treating provider, making it difficult to compare rates between studies.
Choice of vasopressor did not impact rates of prolonged (greater than 48 h) mechanical ventilation. These results mirror other meta-analyses of patients with septic shock, where duration of mechanical ventilation (MD -0.58 h, 95%CI -1.47, 0.31) or number of ventilator-free days (13 vs 13) was not different between vasopressin and other pressors[26,34].
This systematic review has several limitations which should be highlighted. A large portion of our literature search met exclusion criteria because of study design or intervention which limits the sample size available for analysis. Of the studies included, only two reported many of the outcomes of interest, further limiting sample size. The studies also differ in methodology and risk of bias, making comparison of results between studies more challenging. There was also significant variation in dosing strategies of vasopressin and the reporting of concurrent vasopressor therapy which likely impacted results. This, combined with the heterogeneity revealed between the studies, reduce the reliability of the reported results.
Patients who experience vasoplegic shock suffer from significant morbidity and mortality and identification of optimal treatment modalities is of paramount importance to clinicians caring for these patients. Given its unique pharmacology, vasopressin may play a role as optimal therapy in certain patients with vasoplegic shock but should be considered as adjunct in all patients refractory to catecholamines. While current literature is promising, several questions still remain about vasopressin, such as ideal dosing strategies, timing of initiation, and in which patient populations vasopressin as a primary pressor may be ideal. Additional prospective multi-center research is warranted to investigate vasopressin’s role in improving patient-centered outcomes of post-operative vasoplegic shock on a large scale.
Vasoplegic shock is a devastating complication post-surgery, in particular cardiac surgery, that leads to poor patient outcomes. Currently, treatment for this condition consists of norepinephrine and epinephrine. However, because of vasopressin’s unique pharmacology, it may have a role in the treatment of this condition.
Effective therapies aimed at hemodynamic preservation have not been identified in vasoplegic shock. Although norepinephrine and epinephrine are routine management, they have not proven all that effective for this condition given their hemodynamic profile and association with other complications. Vasopressin with its unique pharmacology and beneficial association with certain patient centered outcomes may be a reasonable first line alternative.
The aim of this systematic review was to summarize the available literature evaluating vasopressin vs non-vasopressin alternatives on patient-centered outcomes of vasoplegic shock in adult intensive care unit (ICU) patients. The aim of the present study will provide useful information on whether vasopressin maybe beneficial in the treatment of vasoplegic shock.
Randomized controlled trials, prospective cohorts, and retrospective cohorts comparing vasopressin to norepinephrine, epinephrine, methylene blue, hydroxocobalamin, or other pressors were included. The primary outcomes of interest were 30-d mortality, atrial/ventricular arrhythmias, stroke, ICU length of stay, duration of vasopressor therapy, incidence of acute kidney injury stage II-III, and mechanical ventilation for greater than 48 h. Given the mixed methodologies and heterogenous populations of the included studies and the overall small sample size, a meta-analysis was not conducted. We present weighted mean difference for continuous outcomes and relative risk for binary outcomes with associated confidence intervals.
A total of 1161 studies were screened for inclusion with 3 meeting inclusion criteria with a total of 708 patients. Two studies were randomized controlled trials and one was a retrospective cohort study. Primary outcomes of 30-d mortality, stroke, ventricular arrhythmias, and duration of mechanical ventilation were similar between groups. Conflicting results were observed for acute kidney injury stage II-III, atrial arrhythmias, duration of vasopressors, and ICU length of stay with higher certainty of evidence in favor of vasopressin serving a protective role for these outcomes. Although our results do not provide conclusive evidence of a beneficial role for vasopressin in the treatment of vasoplegic shock, we do provide some rationale as to why vasopressin could have a protective effect with regards to certain patient centered outcomes such as acute kidney injury, atrial arrhythmias, etc. We also provide some direction for future research in this area.
Vasopressin was not found to be superior to alternative pressor therapy for any of the included outcomes. Results are limited by mixed methodologies, small overall sample size, and heterogenous populations. We identify limitations in the present systematic review such as mixed methodologies and heterogeneous populations that preclude a definitive answer on the role of vasopressin in vasoplegic shock. Future studies should have more homogenous populations with similar methodologies so that a pooled analysis can be performed to definitively answer this question.
While current literature is promising, several questions still remain about vasopressin, such as ideal dosing strategies, timing of initiation, and in which patient populations vasopressin as a primary pressor may be ideal. Additional prospective multi-center research is warranted to investigate vasopressin’s role in improving patient-centered outcomes of post-operative vasoplegic shock on a large scale taking into consideration dosing strategies and timing of initiation of vasoactive agents.
Argenziano M, Choudhri AF, Oz MC, Rose EA, Smith CR, Landry DW. A prospective randomized trial of arginine vasopressin in the treatment of vasodilatory shock after left ventricular assist device placement.Circulation. 1997;96:II-286.
Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, Cates CJ, Cheng HY, Corbett MS, Eldridge SM, Emberson JR, Hernán MA, Hopewell S, Hróbjartsson A, Junqueira DR, Jüni P, Kirkham JJ, Lasserson T, Li T, McAleenan A, Reeves BC, Shepperd S, Shrier I, Stewart LA, Tilling K, White IR, Whiting PF, Higgins JPT. RoB 2: a revised tool for assessing risk of bias in randomised trials.BMJ. 2019;366:l4898.
Wells G, Shea B, O'Connell D, Peterson J, Welch V, Losos M, Tugwell P.
The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2012 [cited 2020 September 24]. Available from: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.