Exercise, Mental Health, and Big Data

Recently, a large cross-sectional study that investigated the relationship between exercise frequency and mental health was published in The Lancet Psychiatry and also happened to set Twitter on fire. I want to discuss the good and the not so good. First, some good!

Some Good

The study, which included 1,237,194 adults from the US, found a significant relationship between physical exercise and self-reported mental health burden.

Here’s what the authors reported,

“In an analysis of 852,068 matched individuals (627,510 [73.6%] of whom exercised), we found that exercise was associated with a 1.49 day (43.2%) lower self-reported mental health burden per month (W = 7.42×10¹⁰, p < 2.2×10– ¹⁶) for individuals who exercised than those who did not.

We repeated the same analysis specifically for matched individuals with (110,194; 71,111 [64.5%] of whom exercised) and without (741,874; 556,399 [75.0%] of whom exercised) a previous depression diagnosis. The effect was larger among individuals who reported a previous depression diagnosis, in whom exercise was associated with a 3.75 day (34.5%) lower mental health burden (W=1.61×10⁹, p < 2.2×10– ¹⁶), than among people who reported a previous diagnosis of depression but did not exercise.“

They also found this relationship between high exercise frequency and mental health,

“We also observed U-shaped relationships between exercise frequency and mental health burden, whereby individuals who exercise between three and five times a week had a lower mental health burden than those who exercised fewer than three or more than five times (appendix p 46).“

Graphical results from the big data exercise cross sectional study published in The Lancet

Some Not So Good

Wow! That’s neat. I guess this study shows that moderate amounts of exercise can indeed improve mental health and that too much of it is harmful… right? Well, that is what the media ended up reporting.

How the media ended up reporting the results of this big data cross sectional exercise study, often with very strong causal language

However, because this is a cross-sectional study, AKA a snapshot in time, it is hard to know which came first. Did the exercise cause the positive mental health states or did the positive mental health states cause the activity? Unfortunately, these types of studies can’t answer this question, but longitudinal studies and randomized controlled trials, where we can observe participants over time, can give us some answers.

So what does that evidence say? Meta-analyses of randomized trials seem to find that exercise has large and positive treatment effects on mental health outcomes such as depression.

Study Name # of RCTs Effect + 95% CI
Schuch et al. 2016 25 1.11 (0.79-1.43)
Gordon et al. 2018 33 0.66 (0.48, 0.83)
Krogh et al. 2017 35 −0.66 (-0.86, -0.46)

But, when you only pool high-quality studies, the effects become tiny.

“Restricting this analysis to the four trials that seemed less affected of bias, the effect vanished into −0.11 SMD (−0.41 to 0.18; p=0.45; GRADE: low quality).” - Krogh et al. 2017

The authors, acknowledging mainly the good of these previous studies, suggest this in their conclusion,

“It is plausible that these findings are causal because they are consistent with randomised controlled trials indicating positive effects of exercise on mental health outcomes, anxiety, and post-traumatic stress disorder, as well as cohort studies suggesting that exercise protects against the incidence of depression.”

Some Good

They also did acknowledge the limitations of their study,

“The cross-sectional nature of this study, however, limits the ability to establish the direction of causality for the association between exercise and mental health. Although this concern is partially mitigated by randomised controlled trials indicating positive effects of exercise on mental health, at least one longitudinal observational study has suggested that the association might be bidirectional. In other words, inactivity might be both a symptom of and contributor to poor mental health, whereas activity might be an indicator of and contributor to resilience. “

It is possible that several science journalists may have taken the author’s words for granted (where they suggested causality) without focusing so much on the limitations of cross-sectional studies and the problem of reverse-causality and confounding.

However, it wasn’t just the media’s shoddy reporting that seemed to have set Twitter on fire when this study made the rounds. It was also because one of the authors of the study was a co-founder of a startup, Spring Health, which described its mission as,

To use leading technology to help people feel better—faster.

All the authors extensively disclosed their conflicts of interest in the paper. As for the co-founder of Spring Health, here’s what the paper said,

“AMC [the author/cofounder] holds equity in Spring Care. He is lead inventor on three patent submissions relating to treatment for major depressive disorder (USPTO docket number Y0087.70116US00, USPTO provisional application number 62/491,660, and USPTO provisional application number 62/629,041). He has consulted for Fortress Biotech on antidepressant drug development”

Fair enough. The authors were completely transparent. So why were people still on fire about this study?

Some Not So Good

The author had suggested that the results of this study would be used to help personalize exercise plans for depressed individuals, despite the limitations of it.

Furthermore, the startup, Spring Health (which has a Twitter page), retweeted all of the studies that suggested in their headlines that exercised improved mental health.

More flawed newspaper headlines

So while the authors were very clear about the limitations of their study in their paper, and even briefly on Twitter, they also suggested that this data could be used to help personalize exercise for patients, despite the study being a snapshot in time. The authors also allowed Spring Health to retweet several misleading headlines. This is problematic because it is science journalists and the media that influence policy. If science journalists are not skeptical and misreport studies, they have the potential to a lot of damage. That sort of damage is tough to undo.

I would also recommend reading this excellent article, "Sensationalism in the media: when scientists and journalists may be complicit collaborators."

Here’s a nice excerpt,

“Because democracies rely on an informed citizenry to debate and decide among policy choices, sensationalism may threaten effective involvement by desensitizing the public to information about medical science through repetitive cycles of excitement and disappointment.”

That’s all for today folks.

  • Cite this blog post

  • See also:

    comments powered by Disqus