I greatly enjoyed Mohamed Gad−el−Hak's Opinion piece "Publish or Perish—An Ailing Enterprise?" (Physics Today, March 2004, page 61). A physicist by undergraduate training, I am now doing auditory research in psychology. It is my sad duty to report that the proliferation of publishing mediocrity that Gad−el−Hak describes with such clarity is hardly limited to the physical sciences. Unfortunately, psychology is dominated by the current political correctness, so if anyone is to lead us out of this publishing wasteland, it will not be a psychologist. It is good, therefore, that a physical scientist has the courage to say what needs to be said. I have openly expressed my own discontent with some aspects of the current scientific system1 but my comments fell on deaf ears.
Gad−el−Hak's briefly worded solution to the problems of evaluating both journals and individual authors is a familiar one: impact factors, judged by citation rate. He immediately acknowledges, however, that sheer numbers of citations may not be a fair "index of competence," as he puts it, for younger scientists; he suggests the number of citations per publication as an alternative. That, too, has its problems.
Impact, though, has nothing to do with competence. Rating the impact of a journal is a different task from rating the competence of an individual. The effects of the competence or incompetence of individual papers average out to produce a greater or lesser reputation for a given journal. As the journal matures, its reputation stabilizes, and can even improve.
The impact of a young scientist is not a sensible concept, especially given that "young" refers to something completely different from what it did, say, 30 years ago.2 In 1970, for example, the average age at PhD matriculation in this country might have been 26. Today, it is much likely closer to 35. Granted, some precocious individuals have had an unmistakable impact by obtaining numerous citations at a comparatively young age. But for the majority of younger scientists, citation count is not a mark of competence.
Citations can be given in a prejudicial fashion. For example, there are tales of citation cartels in which people, research groups, or even institutions agree to favor each other's work. I believe I know of several in my field. A lack of citation can also be due to personal acrimony. Furthermore, some work is not acknowledged simply to avoid bolstering the author's citation count!
These factors, combined with the sheer volume of published work, can prevent even first−rate work from being noticed. In such an atmosphere, only written evaluations by those who have read the candidate's work can be taken as formal indicators of competence. But this approach runs head−on against another problem identified by Gad−el−Hak: profligate coauthorship. Exactly whose work is to be evaluated? Someone can easily be a coauthor of a well−cited paper to which he or she has contributed little insight. How do we know for sure whose impact is being factored?
Gad−el−Hak has done us all a favor by so eloquently pointing out what has happened to academic publishing under the impact of publish−or−perish.
For measuring a scientific journal's quality, Mohamed Gad−el−Hak supports the impact factor, which he defines as "the total number of citations made in that year for articles published in the two preceding years divided by the number of citable articles published in those years." A weakness of this factor is that the period between a paper submission and its publication is typically one year, and the two−year period is actually halved. On the other hand, simply noting the weakness may stimulate editors to shorten the publication period.
The major shortcoming of assessing scientific productivity by both the number of publications and the number of citations is that such an assessment is typically made without regard to the number of coauthors. This means that a publication's weight is proportional to the number of coauthors, which looks absurd. It would be more reasonable to assign 0.5 point to the first author and divide the other 0.5 point among the remaining authors. Another, simpler way to assess an individual is to consider the total number of the person's publications and citations along with the number in which that person is the first or only author.
Gad−el−Hak suggests reducing a person's list of publications to 5−10 significant papers. In my view, it is first necessary to remove all abstracts, conference talks, proceedings, and other unrefereed publications from the listing. Doing so would shorten many lists by a factor of two or three.
The number of citations for a published paper depends strongly on how wide the particular field of science is. I work in planetary science, which includes very different fields like geology, atmospheric science, and magnetospheric studies—fields that rarely overlap. Planetary scientists study approximately 20 main bodies (planets and major satellites), asteroids, comets, and interplanetary medium. A paper on, for example, the Martian atmosphere typically would not cite publications on the atmospheres of other planets.
The American Astronomical Society's division for planetary sciences has about 1200 members. Approximately 80% of their publications are in two journals, Icarus and the Journal of Geophysical Research: Planets. These two journals published a total of 418 papers in 2003; the publication rate for members is rather low, approximately 0.45 paper per person per year. But mean number of authors is four per paper, which increases the rate to approximately 1.8. Evidently, citation indices of people who study, say, nitrogen−methane atmospheres of Titan, Triton, and Pluto are much lower than those who study, say, black holes. Therefore, papers in planetary science do not appear in listings of the most cited astronomy and astrophysics publications.
One could argue that the publish−or−perish system is now working to the disadvantage of science: The noise is drowning out the signal. Mohamed Gad−el−Hak makes good suggestions for alternative, more quality−sensitive ways than raw paper count to measure one's effective contribution.
I suggest that quality will quickly improve at the expense of volume if academic departments or journals or both properly recognize the task of refereeing. I believe that refereeing should be elevated to a status approaching that of paper authorship; the contribution to science of a good referee's report identifying some fundamental error and barring publication is greater, perhaps, than the contribution of several mediocre published papers. To implement such elevated status, departments could simply agree to count the refereeing beans along with those for authorship. And journals could publish referees' names along with an accepted paper—does anonymity really exist in the present system? Referees rejecting papers should also be acknowledged somehow.
Gad−el−Hak replies: From these and many other Letters I received, it appears that there is a consensus on the crux of my Opinion piece. We merely need to mull over some of the details.
Lance Nizami's points regarding youth, citation cartels, and personal acrimony are well taken. Nevertheless, I still think that counting a normalized number of citations combined with the opinions of several experts is the least subjective way of judging the worthiness of one's publications. I have no simple answer for another good point Nizami tackles: "Exactly whose work is to be evaluated?"
Notwithstanding Vladimir Krasnopolsky's opening sentence, I do not take credit for defining the established impact factor, but I agree that two years is insufficient time to accumulate a representative number of citations, considering that one of those two years does not exist in practice. The important thing is to use a standard definition that applies equally to everyone, whether the writer's field has many journals or only a few. How I would like to be in the field of planetary science, in which 80% of the publications appear in two journals, in contrast to the 250 journals in fluid mechanics! Still, we ought not to change the definition or the rules to fit the discipline.
I agree with Michael Ibison's suggestion that quality may improve if we all give sufficient credit to the task of refereeing. However, the reviewers' anonymity, though difficult to uphold, does have considerable value, and I do not have a simple solution to the obvious contradiction. Although far from being infallible, the entire enterprise of citation index and impact factor is better than the alternative, straightforward bean counting.
Although generally favorable, none of the Letters I received directly addresses the crux of the problem: How do we put the brakes on the growth rate of journal and book pages? Yes, we should strive for quality, but in order to drive down the demand and hence the supply for journal pages, we must tame our unrealistic expectations of anyone who is up for tenure or promotion. My own minuscule, nonscientific survey revealed two points. First, scientists in the trenches support limiting the number of publications in any resumé submitted to a hiring, tenure, or promotion committee or to a funding agency. Second, university administrators—the ones who can make and enforce such decisions—unfortunately have shown no interest in the discussion.
Featured Jobs
