Why Medical Studies Are Often Wrong

Aug. 7, 2005 — -- How many times have you heard people exclaim something like, "First they tell us this is good or bad for us, and then they tell us just the opposite"?

In case you need more confirmation for the "iffy-ness" of many health studies, Dr. John Ioannidis, a researcher at the University of Ioannina in Greece writing in the Journal of the American Medical Association, recently analyzed 45 well publicized studies from major journals appearing between 1990 and 2003. His conclusion: the results of approximately one third of these studies were flatly contradicted or significantly weakened by later work.

There's the well-known story of hormone replacement therapy, which was supposed to protect against heart disease and other maladies, but apparently does not. A good part of the apparent effect may have been the result of attributing the well-being of upper middle class health-conscious women to the hormones.

Another bit of health folklore that "everybody knows" that has turned out to be unfounded is vitamin E's protective effect against cardiac problems. Not so says a recent large study.

And how about red wine, tea, fruits and vegetables? Surely the anti-oxidant effect of these wondrous nutrients can't be doubted. Even here, however, the effect appears to be more modest than pinot noir lovers, among others, had thought.

And certainly many lung patients who inhale nitrous oxide and swear by its efficacy will be surprised to learn that a larger study does not show any beneficial effect.

A common procedure to remove fat from neck arteries, prescription drugs used by millions of people, the herb echinacea … The examples extend beyond those in the JAMA article and go on and on, but the general point is that a single health study by itself cannot be taken as indubitable. The totality of the available evidence, appropriately weighted, is what counts, and this balanced appraisal is difficult to fit into a news article, much less into a catchy headline.