Tuesday, May 02, 2006

the holes in peer review

New York Times science writer and physician Lawrence K. Altman today takes scientific peer review of journal articles to task in a commentary that reviews most of the steps in the peer review process -- and most of the holes in it, too. News about journal-published findings are among the most popular types of science news, and nonscientists have come to look for the term "peer-reviewed" as a marker of quality. Not so, says Altman:

However, even the system's most ardent supporters acknowledge that peer review does not eliminate mediocre and inferior papers and has never passed the very test for which it is used. Studies have found that journals publish findings based on sloppy statistics. If peer review were a drug, it would never be marketed, say critics, including journal editors.

Altman's commentary includes a rundown of recent retractions of findings that went through peer review, but later turned out to be fraudulent. If you're releasing scientific news, you should expect -- and welcome -- questions that go beyond "is it peer reviewed?" when establishing the quality of the research. Altman's commentary provides a useful guide to consult in understanding the process and what it can -- and can't -- do.

No comments: