When Can We Say That Something Doesn’t Work?

People don’t want to waste their time on things that don’t work. To avoid wasting time, many may want to assess the scientific evidence. They may first look at the basic science (if it can be studied at such a level) and ask, “does this thing have a clear molecular/biological mechanism,” or they may ask, “does it have a theoretical foundation?” Next, the person may look at the human evidence (if there is any) and ask if it worked in a clinical trial or epidemiological data. Read More

High Statistical Power Can Be Deceiving

Even though many researchers are now acquainted with what power is and why we try to aim for high power in studies, there are still several misconceptions about statistical power floating around. For example, if a study designed for 95% power fails to find a difference between two groups, does that offer more support for the null hypothesis? Many will answer yes, because they elicit that if such a large study failed to find a difference between two groups, then this provides evidence for no effect. Read More

Meta-Analysis: Choose Your Model Wisely

Why Do a Meta-Analysis? Meta-Analyses are statistical techniques where we take the treatment effects from several studies and pool them to estimate the overall treatment effect. This type of analysis is conducted so that we can get a precise estimate of the true treatment effect or estimate the mean of several true treatment effects and to notice how robust the effects are across studies. If the individual study treatment effects tend to vary substantially across the studies (true heterogeneity, separated from random error), then we want to explore the sources of this observed dispersion. Read More

Myth: Covariates Need to Be Balanced in RCTs

Before I get into the nitty-gritty, I want to remind everyone why we randomize in the first place. It’s to reduce selection bias and to get rid of systematic variation amongst groups, which allows us to come to more precise and efficient causal inferences. Many critics claim that we can’t make valid causal inferences if there’s an imbalance in covariates between the groups. Here’s an example. Say, hypothetically, we had two groups, and we wanted to see the effect of a statin on all-cause mortality and compare it to placebo. Read More

Four Misconceptions About Statistical Power

Statistical power, within the context of hypothesis testing, is the probability of rejecting the test hypothesis at a specified level, given that the alternative hypothesis is true. In simpler words, it’s the probability of finding a statistically significant effect, when there is one. Here, I address some misconceptions I often see about statistical power. Misconception: Statistical power can only be increased with larger sample sizes Truth: Nope. Here are some other things that increase statistical power: Read More