Why Medical Studies Are Often Wrong

In the other direction, people often over-react to bad news and fall subject to the "tyranny of the anecdote." For example, TV viewers see parents keening about the unfortunate effect of some vaccine on their child and give little weight to the hundreds of thousands of children who've benefited from the same vaccine.

A distinction from statistics is marginally relevant. We're said to commit a Type I error when we reject a truth and a Type II error when we accept a falsehood. In listening to news reports people often have an inclination to suspend their initial disbelief in order to be cheered and thereby risk making a Type II error. In evaluating medical claims, however, researchers generally have an opposite inclination to suspend their initial belief in order not to be beguiled and thereby risk making a Type I error. There is, of course, no way to always avoid both types of error, and we have different error thresholds in different endeavors.

Moreover, the questions health studies address are often subtly different so seemingly contradictory or confirmatory results are difficult to compare and evaluate. Also sobering is the realization, acknowledged by the JAMA author Ioannidis, that there's no conclusive proof that the results of later studies will not also be rescinded or modified.

So what should you conclude about, say, a small new study that flavonoids in dark chocolate help lower blood pressure? It's your call, but realize how credible you find this chocolate study may say more about your psychology than the biochemistry of chocolate.

As I've written before (although with a different number), it's been conclusively established that 43.58871563% of all statistics are made up on the spot.

-- Professor of mathematics at Temple University, John Allen Paulos is the author of best-selling books, including "Innumeracy" and "A Mathematician Plays the Stock Market." His "Who's Counting?" column on ABCNews.com appears the first weekend of every month.

-- This embed didnt make it to copy for story id = 997688.
  • 1
  • |
  • 2
Join the Discussion
blog comments powered by Disqus
You Might Also Like...