Tag Archives: Peer review

Is peer review biased against women?

In astrophysics, time-allocation committees allocate the time available on major telescopes and satellites. Facilities such as the Hubble Space Telescope are often over-subscribed by factors of 4 to 10, and writing proposals that win time in peer review is crucial to people’s careers.

I’ve recently had my first experience of serving on a “dual-anonymous” time-allocation committee, in which all the proposals are reviewed without knowing the proposers’ names.

The trend to dual-anonymous review started with an analysis of 18 years of proposals to use Hubble. The crucial data showed that, over more than a decade, the success rate for proposals with a woman as principal investigator was lower (at 19%) than for proposals led by men (at 23%). That 20% difference in success rate then disappeared when proposals were reviewed without knowing the proposer’s names. Continue reading

What is the point of scientific peer review?

peer-review-thumbnailIn order to publish in any scientific journal worth publishing in, ones paper must pass “peer review”. This consists of the editors asking one or more experts in the field to review the paper in order to spot errors, recommend improvements, and opine on whether it is of sufficient merit to publish.

Richard Smith, former editor of the British Medical Journal, has written an article in the Times Higher Education arguing that “the process has little if any benefit and lots of flaws”, and that “peer review doesn’t work” because most of what is published in scientific journals “is plain wrong”.

He tells the story, when at the BMJ, of deliberately inserting multiple errors into manuscripts to see how many reviewers found them. Many of the reviewers missed many of the errors.

This raises the question of what peer review is actually for. Smith states that “Peer review is supposed to be the quality assurance system for science”, that aims to reassure readers “that they can trust what they are reading” (added emphasis).

However, no working scientist actually “trusts” scientific papers, all scientists read papers in a sceptical way, knowing full well that much of what they are reading will turn out to be wrong. “The” quality-assurance system for science is not what happens prior to publication, but what happens after publication. That is when the real peer review begins, when the community can read and think about and criticise a paper. Continue reading