Bias Can Exaggerate Drugs' Effectiveness

Doctors - and patients - might not be getting all the information they need about the safety and effectiveness of certain drugs because of "publication bias," the tendency of researchers and medical journals to favor positive results over negative ones.

Researchers running drug trials are required to submit detailed results to the U.S. Food and Drug Administration. But when it comes to reporting trial results publicly in medical journals, "it's an entirely different ballgame," according to Dr. Erick Turner, lead author of a study published today in PLoS Medicine.

"Doctors are trained to regard medical journals as the gospel truth," said Turner, assistant professor of psychiatry and pharmacology at Oregon Health and Science University in Portland. "But what we're learning here is it's not necessarily the whole truth and nothing but the truth."

Turner and colleagues reviewed the results submitted to the FDA for eight antipsychotic drugs used to treat schizophrenia. They then compared the results to those published in medical journals. Four trials submitted to the FDA, all of which had unflattering results for the drug under study, were unpublished.

"It's kind of like grade inflation," said Turner. "Say you've got a class full of kids. Some are excellent students and some should be failing. If you give everyone an A, an outside person is not going to be able to appreciate there's a difference between these two sets of kids."

Even when the studies were published, the journal articles often overemphasized the drugs' effectiveness.

"Some of what we found could constitute spin, some would fall into the category of shenanigans," said Turner. "The take-home message is there are loopholes in the publication process by which doctors may be relying on information that's incomplete or somehow skewed. The drug's effects may be exaggerated or its safety concerns may be downplayed."

Turner said researchers working for a drug company might be inclined to withhold data that's seen as damaging, adding "there's no law says they have to publish." But certain medical journals are also less likely to accept negative trials for publication. Bias ,Turner said, could be mitigated by leaving the results out of the decision to publish.

"The person reviewing the study for the journal should be thinking, 'Is this good science?' rather than basing a decision on the results," he said.