Return to site

Observational Studies Vs Interventional Studies: Even Doctors Get This Wrong

Oliver Zolman MD

· Health Evidence

Lesson 1: Its fundamental to know the difference between observational studies and interventional studies

Observational studies are those where you "observe" what happens to people's health without doing anything, like looking at animals in the zoo and tracking things they do, like their diet, exercise, drugs they take, genes etc.

"Interventional studies" are those where you make people do something and test the results, like giving half the monkeys bananas and measuring if they get diabetes compared to the monkeys that don't have bananas.

Lesson 2: Observational studies can't prove something is good or bad for you, unless you correct the results for every possible confounding factor

You look at the population in the observational study and find that those that eat more vitamin E have less heart disease.

But this study wasn't corrected for every possible confounding factor. In most cases its impossible to even know all the confounding factors.

Then an interventional study testing vitamin E for heart disease prevention finds that it has no effect on risk of heart disease.

Lesson 3: Whenever you seem health claims online, ask "is this based on interventional studies"?

This study (http://www.dcscience.net/Young-Karr-2011.pdf) showed 52 health claims from observational studies that suggested benefit from doing a certain health behaviour were all WRONG when they did interventional studies to confirm them. 10% of studies actually showed harm rather than benefit.

None of these studies were adjusted for all possible confounding factors.

Based on this study you're more likely to avoid harm if you do the opposite of what observational studies show (but this study was only 52 health claims out of hundreds of thousands of observational health study claims, so one cannot generalise this point).

Interventional studies such as non-majorly flawed randomised controlled trials (RCTs), trump observational studies for accuracy of health claims the vast majority of the time as you can prove that something causes good or bad health.

Lesson 4: Observational studies can rarely be useful if the results are corrected for all possible confounding factors, like with observational studies of smoking

When it was observed that smokers died younger and had lung cancer and heart disease at much higher rates than non-smokers, the difference was so much that there was no other plausible cause of these diseases other than smoking.

Similarly, say you look at the whole population and you sequence all the genes of everyone, and you find that everyone that has a certain gene mutation has a certain disease and everyone without this mutation never has the disease, this can be sufficient evidence to give high likelihood of causation from an observational study.

Or say you look at the population of people getting diarrheoal illness and you find the only people getting sick are those that drink polluted water from a particular source, this is quite good evidence of causation.

Lesson 5: Use my observational study checklist

Checklist for using observational evidence to guide health behaviours to overcome bias from residual confounding and reverse causation

  • Mendelian randomisation evidence?
  • Is multi-variate confounder adjustment sufficient and is it not over-adjusted?
  • Dose response relationship? 
  • Multi-long term outcome meta-analyses of individual participant data?
  • Multiplicity correction?
  • Sufficiently large 95% CI closest to 1 bound effect sizes (i.e. at least 1.5 RRI (relative risk increase) or 0.66 RRR (relative risk reduction))?
  • Supported by validated low attrition and allocation bias RCT surrogate markers or shorter term clinical outcomes?
  • Other causative mechanistic evidence e.g. N=1?
  • Directness of PICOTS?
All Posts
×

Almost done…

We just sent you an email. Please click the link in the email to confirm your subscription!

OK