Recently, a large cross-sectional study that investigated the relationship between exercise frequency and mental health was published in The Lancet Psychiatry and also happened to set Twitter on fire. I want to discuss the good and the not so good. First, some good!
The study, which included 1,237,194 adults from the US, found a significant relationship between physical exercise and self-reported mental health burden.
Here’s what the authors reported,
“In an analysis of 852,068 matched individuals (627,510 [73.6%] of whom exercised), we found that exercise was associated with a 1.49 day (43.2%) lower self-reported mental health burden per month (W = 7.42×10¹⁰, p < 2.2×10– ¹⁶) for individuals who exercised than those who did not.
We repeated the same analysis specifically for matched individuals with (110,194; 71,111 [64.5%] of whom exercised) and without (741,874; 556,399 [75.0%] of whom exercised) a previous depression diagnosis. The effect was larger among individuals who reported a previous depression diagnosis, in whom exercise was associated with a 3.75 day (34.5%) lower mental health burden (W=1.61×10⁹, p < 2.2×10– ¹⁶), than among people who reported a previous diagnosis of depression but did not exercise.“
They also found this relationship between high exercise frequency and mental health,
“We also observed U-shaped relationships between exercise frequency and mental health burden, whereby individuals who exercise between three and five times a week had a lower mental health burden than those who exercised fewer than three or more than five times (appendix p 46). “
Some Not So Good
Wow! That’s neat. I guess this study shows that moderate amounts of exercise can indeed improve mental health and that too much of it is harmful… right? Well, that is what the media ended up reporting.
However, because this is a cross-sectional study, AKA a snapshot in time, it is hard to know which came first. Did the exercise cause the positive mental health states or did the positive mental health states cause the activity? Unfortunately, these types of studies can’t answer this question, but longitudinal studies and randomized controlled trials, where we can observe participants over time, can give us some answers.
So what does that evidence say? Meta-analyses of randomized trials seem to find that exercise has large and positive treatment effects on mental health outcomes such as depression.
of Randomized Trials
|Effects (SMD) + Confidence Intervals|
|Schuch et al. 2016||25||1.11 (95% CI, 0.79-1.43)|
|Gordon et al. 2018||33||0.66 (95% CI, 0.48-0.83)|
|Krogh et al. 2017||35||−0.66 (95% CI, -0.86, -0.46)|
But, when you only pool high-quality studies, the effects become tiny.
“Restricting this analysis to the four trials that seemed less affected of bias, the effect vanished into −0.11 SMD (−0.41 to 0.18; p=0.45; GRADE: low quality).” - Krogh et al. 2017
The authors, acknowledging mainly the good of these previous studies, suggest this in their conclusion,
“It is plausible that these findings are causal because they are consistent with randomised controlled trials indicating positive effects of exercise on mental health outcomes, anxiety, and post-traumatic stress disorder, as well as cohort studies suggesting that exercise protects against the incidence of depression.”
They also did acknowledge the limitations of their study,
“The cross-sectional nature of this study, however, limits the ability to establish the direction of causality for the association between exercise and mental health. Although this concern is partially mitigated by randomised controlled trials indicating positive effects of exercise on mental health, at least one longitudinal observational study has suggested that the association might be bidirectional. In other words, inactivity might be both a symptom of and contributor to poor mental health, whereas activity might be an indicator of and contributor to resilience. “
It is possible that several science journalists may have taken the author’s words for granted (where they suggested causality) without focusing so much on the limitations of cross-sectional studies and the problem of reverse-causality and confounding.
However, it wasn’t just the media’s shoddy reporting that seemed to have set Twitter on fire when this study made the rounds. It was also because one of the authors of the study was a co-founder of a startup, Spring Health, which described its mission as,
To use leading technology to help people feel better—faster.
All the authors extensively disclosed their conflicts of interest in the paper. As for the co-founder of Spring Health, here’s what the paper said,
“AMC [the author/cofounder] holds equity in Spring Care. He is lead inventor on three patent submissions relating to treatment for major depressive disorder (USPTO docket number Y0087.70116US00, USPTO provisional application number 62/491,660, and USPTO provisional application number 62/629,041). He has consulted for Fortress Biotech on antidepressant drug development”
Fair enough. The authors were completely transparent. So why were people still on fire about this study?
Some Not So Good
The author had suggested that the results of this study would be used to help personalize exercise plans for depressed individuals, despite the limitations of it.
Where do we go from here? Over [@spring_health](https://twitter.com/spring_health?ref_src=twsrc%5Etfw) - our mental health startup in New York City - we’re using these findings to develop personalized exercise plans. We want to help every individual feel better—faster, and understand exactly what each patient needs the most. pic.twitter.com/egSgBVY8Fm
— Adam Chekroud (@itschekkers) August 11, 2018
Furthermore, the startup, Spring Health (which has a Twitter page), retweeted all of the studies that suggested in their headlines that exercised improved mental health.
Or - god forbid - newspaper headlines trumpeting the flawed causal assumptionhttps://t.co/shp7dxu5sR
More flawed newspaper headlineshttps://t.co/sjtEWmFwJA
So while the authors were very clear about the limitations of their study in their paper, and even briefly on Twitter, they also suggested that this data could be used to help personalize exercise for patients, despite the study being a snapshot in time. The authors also allowed Spring Health to retweet several misleading headlines. This is problematic because it is science journalists and the media that influence policy. If science journalists are not skeptical and misreport studies, they have the potential to a lot of damage. That sort of damage is tough to undo.
I would also recommend reading this excellent article, "Sensationalism in the media: when scientists and journalists may be complicit collaborators."
Here’s a nice excerpt,
“Because democracies rely on an informed citizenry to debate and decide among policy choices, sensationalism may threaten effective involvement by desensitizing the public to information about medical science through repetitive cycles of excitement and disappointment.”
That’s all for today folks.