Good stuff: Open Letter to Journal Reviewers.
Dear Reviewer,
Thank you for taking the time to serve as a reviewer for the journal to which I have submitted my paper. Thanks, too, for agreeing to serve as the reviewer of my submission in particular. When we serve as reviewers, we do the profession a great service. The integrity of our profession in large measure depends on competent and conscientious blind review. So, once again, I thank you very much.
However, I have noticed in recent years a marked decline in the quality of the referee reports I have received in response to my journal submissions. Now, of course, maybe the way to explain this is that the quality of my work has declined in recent years. I suppose that’s possible, but I don’t think this could explain the phenomenon– I’ve seen and received high-quality reviews of poor submissions. These are reviews that, despite the ultimate negative judgment regarding the submission, nonetheless do a good job of explaining the weaknesses of the paper, point to definite defects, raise well-targeted objections to actual claims made in the paper, and give a detailed assessment of where the paper’s argument stands vis-a-vis the state of the art in the literature. In short, a high-quality review is a review of the submitted paper, not an opportunity for the reviewer to react to or muse over the paper’s topic.
So I offer a few simple steps that I urge you to consider taking in preparing your review:
1. Give a definite judgment. If the paper is hopeless, say so. If the paper needs significant revision before it could be even in the ballpark of publishibility, say that. If the paper is out of touch with the current literature, say (roughly) what that literature is. If the paper is sound, but not that interesting, say it. And be clear about whether you recommend R&R rather than conditional acceptance. And so on.
2. Before launching into your critical analysis of my paper, provide a paragraph summary of what you take to be its main thesis and argument. This is easy to do, and it’s a great help to the author (I suspect it helps the editor as well). It helps the author to see whether your ultimate judgment regarding the paper is based on a sound reading of it. It helps the author to gauge whether he or she has been clear enough in writing it. Sometimes reviewers reject papers that they misunderstand, and sometimes this misunderstanding is due to the author’s carelessness in framing the paper. Sometimes reviewers misunderstand the paper in such a way that their critical comments are entirely beside the point. Sometimes a R&R decision is based on reviewers who misunderstood the paper; in this case, their suggestions for revision are really suggestions for writing a different paper. And it takes authors a lot of time to figure out how to interpret a reviewer’s comments. A lot of time could be saved if you just state up front what you understand the paper to be about. It’s easy.
3. In giving your critical analysis, you should of course be as forceful as possible, no matter what ultimate decision you recommend. But please be as specific as possible. When you attribute to the author a claim that you think is objectionable, identify the place in the paper where the author makes the claim. And whenever possible, quote what the author actually says. Do not relay on your impressions or your rough sense of what the author claims.
4. Review the paper that the author has written. Whether you believe that some other approach than the one the author has taken is superior is irrelevant. In fact, it’s not clear what one means by terms like “approach,” “methodology,” “discourse,” or “tradition.” It is not clearly a criticism of a paper to say that the author “should consider joining a different conversation.” Nor is it yet a criticism of a paper to say that the author “is dealing with a narrow range of interlocutors.” These remarks are little more than cryptic snobbery unless you say something about how the range of interlocutors is unduly narrow or that the “conversation” that the author is engaging in has been exhausted or proven fruitless. In other words, resist the compulsion to go “meta.” A review for a journal submission is not the place to take out your aggressions concerning the state of the profession. If you can’t do this, you should decline invitations to review.
It seems to me that these steps don’t demand too much. They can be easily satisfied in the usual a three to five paragraph review.
Sincerely,
Author
