Article

Are you skeptical of the latest peer-reviewed results?

Perhaps, like American science fiction writer Theodore Sturgeon, you are skeptical of much that is presented to you as fact. I don't mean this little opinion piece I write each fortnight, which everyone understands to be pure malarkey. No, I am referring to scientific medical publications in our top-tier, peer-reviewed journals.

Well, Greek epidemiologist John Ioannidis, MD, from the University of Ioannina School of Medicine, Ioannina, Greece, offers proof that you are right to be skeptical of what you read. He reviewed 49 "important" research articles published in top medical journals between 1990 and 2003. These important articles were subsequently cited more than 1,000 times by other researchers in their papers. Over time, almost one-third of these papers were found to be wrong.

One refuted study looked into whether hormone replacement therapy was safe for women (first it was, then it wasn't), whether vitamin E protected against heart disease (first it did, then it didn't), and whether stents are better than balloon angioplasty for coronary artery disease (they are, but not as much as originally thought).

Inadequate sample sizes, poor study design, researcher bias (and we all have our biases), financial interests, faulty statistical analyses-all of these were found by Dr. Ioannidis to contribute to studies falsely purporting to reveal new medical truths. Even a large, well-designed study with minimal investigator bias will only give the right answer 85% of the time, according to Dr. Ioannidis' calculations.

Researchers aren't trying to deceive us (in general). But even careful studies performed by well-intentioned medical scientists are imperfect. And let's face the fact that full-time academicians need to publish important papers and get grants if they hope to secure promotions and tenure. So one can't necessarily wait for confirmatory evidence before publishing a "landmark" study that offers a new cause (or cure) for cancer, age-related macular degeneration, or other disease.

According to Dr. Ioannidis, Theodore Sturgeon is wrong; 50% (not 95%) of peer-reviewed biomedical science is crap. One of these two gentlemen is evidently full of, shall we say, baloney.

The problem is that medical practice is not a debating society. Changing a patient's medications or performing a different surgical procedure on the basis of a study subsequently proven false is not a good thing.

To me the obvious conclusion is that we doctors should remain skeptical of new claims, even if published in our prestigious peer-reviewed journals. In general, let's wait for the second, confirmatory study from a different group of investigators before changing how we care for our patients on the basis of "exciting" new data.

Ninety-five percent may satisfy Mr. Sturgeon, but not your chief medical editor, who will continue to strive for 100% in this column.

1. Ioannidis JP. Why most published research findings are false. PLoS Med 2005 Aug;2(8):e124. Epub 2005 Aug 30.

2. Scientific accuracy . . . and statistics. Just how reliable are scientific papers? The Economist Sept. 3, 2005.

Peter J. McDonnell, MD is director of The Wilmer Eye Institute, The Johns Hopkins University School of Medicine, Baltimore, and chief medical editor of Ophthalmology Times. He can be reached at 727 Maumenee Building, 600 North Wolfe St., Baltimore, MD 21287-9278 Phone: 443/287-1511Fax: 443/287-1514 pmcdonn1@jhmi.edu

Newsletter

Don’t miss out—get Ophthalmology Times updates on the latest clinical advancements and expert interviews, straight to your inbox.

Related Videos
Abby Markward, MBA, and Hattie Hayes, editor of Ophthalmology Times Europe, discuss the ASCRS and ASOA meetings
Abby Markward discusses the ASCRS Foundation and the ASCRS Annual Meeting
(Image credit: Ophthalmology Times) Inside ASCRS 2025: Francis S. Mah, MD, takes the helm with a vision for research, education, and advocacy
(Image credit: Ophthalmology Times) NeuroOp Guru: Cranial nerve six palsy with chemosis is a critical clue to cavernous carotid fistula
(Image credit: Ophthalmology Times) Neda Shamie_Controversies in Modern Eye Care 2025
© 2025 MJH Life Sciences

All rights reserved.