Exercise reduces depressive symptoms in adults with arthritis: Evidential value

Abstract

AIM: To determine whether evidential value exists that exercise reduces depression in adults with arthritis and other rheumatic conditions.

METHODS: Utilizing data derived from a prior meta-analysis of 29 randomized controlled trials comprising 2449 participants (1470 exercise, 979 control) with fibromyalgia, osteoarthritis, rheumatoid arthritis or systemic lupus erythematosus, a new method, P-curve, was utilized to assess for evidentiary worth as well as dismiss the possibility of discriminating reporting of statistically significant results regarding exercise and depression in adults with arthritis and other rheumatic conditions. Using the method of Stouffer, Z-scores were calculated to examine selective-reporting bias. An alpha (P) value < 0.05 was deemed statistically significant. In addition, average power of the tests included in P-curve, adjusted for publication bias, was calculated.

RESULTS: Fifteen of 29 studies (51.7%) with exercise and depression results were statistically significant (P < 0.05) while none of the results were statistically significant with respect to exercise increasing depression in adults with arthritis and other rheumatic conditions. Right-skew to dismiss selective reporting was identified (Z = -5.28, P < 0.0001). In addition, the included studies did not lack evidential value (Z = 2.39, P = 0.99), nor did they lack evidential value and were P-hacked (Z = 5.28, P > 0.99). The relative frequencies of P-values were 66.7% at 0.01, 6.7% each at 0.02 and 0.03, 13.3% at 0.04 and 6.7% at 0.05. The average power of the tests included in P-curve, corrected for publication bias, was 69%. Diagnostic plot results revealed that the observed power estimate was a better fit than the alternatives.

CONCLUSION: Evidential value results provide additional support that exercise reduces depression in adults with arthritis and other rheumatic conditions.

Key Words: $[Keywords]

Core tip: The primary strength of this study was the use of a recent and novel approach to address the potential for selective reporting of statistically significant results, a common problem in the published literature, regarding the effects of exercise on depressive symptoms in adults with arthritis and other rheumatic diseases. The results revealed that selective reporting does not exist, thereby providing further support that exercise improves depressive symptoms in adults with arthritis and other rheumatic diseases.

INTRODUCTION

Arthritis and other rheumatic diseases are a major public health problem affecting more than 52 million adults in the United States[1]. By the year 2030, it is estimated that 67 million Americans 18 years of age and older will have doctor-diagnosed arthritis[2]. In terms of expenditures, the total costs associated with arthritis in the United States were estimated to be 128 billion dollars in 2003, an increase of 41.8 billion dollars compared to 1997[3].

One of the major psychological health problems associated with arthritis and other rheumatic diseases is depression[4]. To illustrate, recent estimates suggest that approximately 18% of United States adults with doctor-diagnosed arthritis have depression[4]. This is the result of people becoming depressed after developing arthritis vs the development of arthritis as a result of being depressed[4].

One potential lifestyle intervention for reducing the prevalence of depression in adults with arthritis and other rheumatic diseases is exercise[5]. For example, a recently completed meta-analysis of randomized controlled trials by the authors resulted in a statistically significant standardized mean difference effect size reduction in depressive symptoms equivalent to a percentile improvement of 16.4 as a result of exercise in adults with arthritis and other rheumatic diseases[5]. While encouraging, all investigations appeared in peer-reviewed academic journals, a potential problem given that publications in academic journals yield an overly excessive number of statistically significant results[6]. Consequently, such findings may not be representative of the truth. Factors associated with an excess of statistically significant outcomes include, but are not necessarily restricted to, selective reporting by researchers[7-13]. Across all levels of utilization, i.e., research, practice, and policy, it is crucial to recognize the genuine consequences of physical exercise on depression in adults with arthritis and other rheumatic conditions. While recommendations for the evaluation of selective reporting and associated biases in meta-analysis have been developed, all have noteworthy shortcomings. As a result, no correction techniques are currently endorsed[14]. However, since the time of publication of these recommendations[14], a new and novel approach known as P-curve has been developed for the purpose of determining whether selective reporting of studies exists and which does not require access to null results[15,16]. Therefore, given the importance of identifying the true effects of exercise on depression in adults with arthritis and other rheumatic conditions, the purpose of the current study was to determine whether there is evidential value that exercise improves depression in adults with arthritis and other rheumatic conditions.

MATERIALS AND METHODS

Literature search

The literature search for the present investigation originated from a previous and recent meta-analysis that has been explained thoroughly elsewhere[5]. In brief, research studies published between 1981 and January 2013 were retrieved by searching ten reference databases, the reference lists of included studies, and expert review.

Study selection

The selection of studies has also been explained thoroughly elsewhere[5]. Succinctly, randomized controlled trials that investigated the effects of aerobic exercise, strength training, or a combination of aerobic and strength training exercise on depressive symptoms, as defined by the authors, in adults with arthritis and other rheumatic diseases (fibromyalgia, osteoarthritis, rheumatoid arthritis, or systemic lupus erythematosus), were included[17-45]. Studies in which exercise, defined as “physical activity that is planned, structured, and repetitive and purposive in the sense that the improvement or maintenance of one or more components of physical fitness is the objective”[46], were included.

Data extraction

The process for data extraction has been described in detail elsewhere[5]. Briefly, data were extracted by both authors, independent of each other. Disagreements were resolved by consensus.

Risk of bias

Risk of bias, described in detail elsewhere, was accomplished using the Cochrane Risk of Bias Assessment Instrument and followed the same procedures as for data extraction[5].

Statistical analysis

The statistical methods of this study were reviewed and approved by a biostatistician, Dr. Matthew Gurka, Department of Biostatistics, West Virginia University.

Outcomes for depressive symptoms, as defined by the authors from each study, were computed using the standardized mean difference effect size. This was calculated by subtracting the change outcome difference in the exercise group from the change outcome difference in the control group, dividing by the pooled standard deviations of the outcomes for both groups, and then weighting them by the reciprocal of the combined variances. All effect sizes were corrected for small sample bias, i.e., Hedges et al[47]. Overall results were then combined using a random-effects model[48]. Heterogeneity and inconsistency were estimated using Cochran’s Q and I² statistic, respectfully[48-50].

To identify whether evidential value exists in relation to exercise reducing depression in adults with arthritis and other rheumatic conditions, the primary purpose of the current study, a recent and novel method known as P-curve was utilized[15,16]. Briefly, the purpose of this approach is to determine whether selective reporting can be excluded as a cause of statistically significant results, thus providing greater confidence that the observed effect is true. It comprises a distribution of significant P-values (alpha level < 0.05) from the included studies. Studies with non-significant P-values (alpha level > 0.05), are excluded from the assessment. The focus of P-curve is on determining whether studies (1) contain evidential value (right skew); (2) lack evidential value, as indicated by a power < 33%; and (3) lack evidentiary importance, i.e., were P-hacked, as indicated by left skew, suggesting that researchers withheld non-significant results. P-results are suggestive of real effects, i.e., evidentiary worth, if the number of small P values (P = 0.01) are greater than the number of large P values (P = 0.04). Testing is twofold. Firstly, for every P-value < 0.05, the chance of detecting a significant P-value at least as excessive as if the null were correct is computed. This P value, i.e., P value of the P value, is computed by dividing each statistically significant probability value from every study by 0.05. With respect to the current investigation, probabilities were calculated using the Z-scores of the differences in depressive symptoms between the exercise and control groups from each included study. To maintain independence, studies that included multiple groups and/or multiple measures of depression using different instruments were combined so that only one probability value was included for that study. This approach was chosen because the focus of this study was on ruling out selective reporting of findings. In addition, P-curve has been found to perform better than previously existing tests to address publication bias[15,16]. Details regarding P-curve have been described in detail elsewhere[15,16].

The second step consists of aggregating PP values using Stouffer’s method[51]. This continuous test is accomplished by computing PP values for each test with a probability of < 0.05 and then converting them to Z-scores. The sum of the Z-scores is then divided by the square root of the number of tests with P-values < 0.05. A negative Z-score and overall P-value < 0.05 is indicative of right-skewed evidential value that results do not suffer from selective reporting bias in favor of statistically significant results. A nonexistent statistically significant right-skewed P-value implies either an absence of data to draw conclusions regarding evidentiary value or a dearth of evidentiary value. To assess for potential absence of data, i.e., power, the identical method as for right-skew is employed with the exception that PP values are recomputed for expected P-curves utilizing a power of 33% along with the sample size from each study, achieved by means of non-central distributions. To test for a lack of evidential value suggestive of the withholding of non-significant findings by investigators, i.e., left skewed P-hacking, the same approach is used as for right-skewed evidential value but the PP values for left skew are computed as 1 minus the right skew PP value. Probability values ≤ 0.05 were considered statistically significant.

In addition to testing for (1) right skew; (2) inadequate information; and (3) left skew, average power of the tests included in P-curve were calculated while correcting for publication bias. This was accomplished by comparing the expected P-curve for each possible value of power between 5% and 99% and then choosing the level of power that most closely matches the expected and observed P-curves.

All data were analyzed using version 3.0 of P-curve (http://www.p-curve.com/app3/), version 3.0 of Comprehensive Meta-Analysis (Englewood, New Jersey, 2015) and Microsoft Excel 2010.

RESULTS

Study selection

Twenty-nine studies that included 2449 participants (1470 exercise, 979 control) with fibromyalgia, osteoarthritis, rheumatoid arthritis, or systemic lupus erythematosus met all eligibility criteria[17-45]. Exercise averaged 19 wk, 4 times per week for 34 min per session[5]. The within-study age of the participants ranged from 18 to 85 years. A detailed description of these studies can be found elsewhere[5].

Changes in depressive symptoms

Figure 1 shows a forest plot of study-level as well as pooled results for changes in depressive symptoms. As shown, the overall results indicate a statistically significant decrease in depressive symptoms in support of exercise along with non-overlapping 95%CI (-0.643--0.298). Heterogeneity was statistically significant (Q = 122.8, P < 0.001) and a large amount of inconsistency was observed (I² = 77.2%, 95%CI = 67.6%-84.0%). Standardized mean difference effect size changes ranged from -1.85 to 0.94. Fifteen of 29 (51.7%) results were statistically significant (P < 0.05) while none were statistically significant with respect to exercise increasing depression in adults with arthritis and other rheumatic conditions.

Figure 1 Forest plot for changes in depressive symptoms. The black squares represent the mean difference while the left and right extremes of the squares represent the corresponding 95%CIs. The middle of the black diamond represents the overall mean difference while the left and right extremes of the diamond represent the corresponding 95%CIs.

P-curve results

Evidential value results are displayed in Table 1 and Figure 2. As shown, there was statistically significant right-skew. This suggests that there is evidential value that exercise decreases depression in adults with arthritis and other rheumatic conditions. Consistent with this finding are the non-significant results for a lack of evidential value, including P-hacking. The average power of the tests included in P-curve, corrected for publication bias, was 69%. Interpretation of the diagnostic plot suggests that the observed power estimate was a better fit than the alternatives (Figure 3).

Table 1 Evidential values for changes in depressive symptoms.

Statistical inference	Z	P
Studies contain evidential value (right-skewed)	5.28	< 0.00011
Studies lack evidential value (flatter than 33% power)	2.39	0.99
Studies lack evidential value and intensely P-hacked (left-skewed)	5.28	0.99

¹Statistically significant (P < 0.05). Negative Z-value for right skew suggests that selective-reporting bias in favor of statistically significant results does not exist. Z: Z-value based on Stouffer’s method; P: Probability value.

Figure 2 P-curve results for evidential value. Results are significantly right-skewed (P < 0.0001), suggesting that evidential value exists that exercise reduces depressive symptoms in adults with arthritis and other rheumatic diseases. The graphed results include 15 statistically significant P-values < 0.05. Fourteen additional results were entered but excluded from the analysis because of non-significance (P > 0.05).

Figure 3 Diagnostic plot for power estimation. This figure illustrates how close the expected P-curve is to the observed P-curve for each level of power between 5% and 99%. The Y-axis is the perfect fit distance for each level of power. The estimated power for exercise-induced changes in depressive symptoms data is 69%. The solid red circle is generally lower than the other markers, suggesting that the power estimate is a better fit than the alternatives. The flatter the curve, the less confidence in the power estimate. Alternatively, a V-shape suggests an ideal estimate of power.

DISCUSSION

Overall findings

The aim of the present investigation was to use a new approach, P-curve, to identify whether evidential value exists in support of exercise for reducing depression in adults with arthritis and other rheumatic conditions. The results suggest there is indeed evidential value in support of exercise aimed at reducing depression in adults with arthritis and other rheumatic conditions. These findings provide additional support to recently completed research on this issue[5]. These findings are noteworthy given: (1) the prevalence of depression in adults with arthritis and other rheumatic conditions[4]; (2) the potential benefits of exercise for improving depression in adults with arthritis and other rheumatic conditions[5]; and (3) the importance of determining if selective reporting bias exists in published exercise studies examining the effects of exercise on depression in adults with arthritis and other rheumatic conditions[7-13].

Implications for research and practice

The findings of the present investigation provide further confirmation regarding the positive effects of exercise on depressive symptoms in adults with arthritis and other rheumatic diseases. However, while a random-effects model that incorporates heterogeneity was used, such models do not explain potential sources of heterogeneity, little of which could be identified in the primary meta-analysis on which the current investigation was based[5]. Given the former, it would appear plausible to suggest that a need exists for well-designed randomized controlled trials to determine what group of participants may benefit the most from exercise. Along those lines, the dose-response effects of exercise were not a purpose of the current study, and when studied previously, did not yield any significant results. Therefore, and as previously recommended[5], there is a need for additional randomized controlled trials in order to determine the dose-response effects of exercise in a representative sample of adults with arthritis and other rheumatic diseases. Until that time, it would appear feasible to recommend that adults with arthritis and other rheumatic diseases progress to achieving the general guidelines of: (1) 150 min per week of moderate-intensity aerobic activity (brisk walking, etc.), 75 min per week of vigorous-intensity aerobic activity (water aerobics, etc.), or some equivalent combination of the two; (2) muscle strengthening exercises at least 2 d per week; and (3) balance exercises at least 3 d per week[52].

Strengths and potential limitations

The primary strength of the present investigation is the use of a new and innovative approach to deal with the issue of potential selective reporting of statistically significant results regarding the effects of exercise on depressive symptoms in adults with arthritis and other rheumatic diseases[15,16]. From the investigative team’s perspective, this is important given the potential for selective-reporting bias and resultant overestimates of beneficial effects found in peer-reviewed journals[7-13]. Alternatively, one possible limitation is that P-curve excludes P values > 0.05 as well as those near 0.05. Consequently, P-values indicative of no effect, while extremely rare when a genuine effect is present, are omitted[16].

The findings of the present investigation provide evidential value regarding the use of exercise for reducing depression in adults with arthritis and other rheumatic conditions. Given the deleterious consequences of depression, exercise should be recommended as a lifestyle intervention for improving depressive symptoms in adults with arthritis and other rheumatic diseases.

COMMENTS

Background

While previous meta-analytic work has demonstrated that exercise improves depressive symptoms in adults with arthritis, the potential for bias, i.e., tendency for statistically significant and positive results to be published, continues to exist.

Research frontiers

There is currently an increased interest in understanding the true effects of exercise on depressive symptoms in adults.

Innovations and breakthroughs

Previous meta-analytic research has demonstrated that exercise improves depressive symptoms in adults with arthritis but the possibility of publication bias cannot be ruled out.

Applications

Using a novel and recently developed approach for assessing publication and other related biases, the results of this study provide additional confirmatory evidence that exercise improves depressive symptoms in adults, thereby providing greater confidence for practitioners when recommending exercise for improving depressive symptoms in adults.

Terminology

Evidential value refers to a lack of publication bias, i.e., tendency for statistically significant and positive results to be published. P-curve refers to a statistical method that assesses whether or not publication and related biases can be ruled out.

Peer-review

In this study the authors introduce a new and novel approach known as P-curve to determine whether selective reporting of studies exists and which does not require access to null results. This is a well-written article with sufficient justification.