Peer reviewing & two solutions

A discussion at Crooked Timber (started here) made me think of a recent editor's note in the American Political Science Review.  First, a snippet of the blog post:

All the editors who took part in the roundtable observe that it is increasingly difficult to find referees. This confirms my experience as an Associate Editor of Feminist Economics, and also reflects the crazy number of requests I get to review papers from all sorts of journals, and also on papers where I strongly doubt I have special expertise. So I’ve been wondering for a long time: is this system sustainable? Is there a way to reward referees, or another way to create positive incentives for refereeing (whether material or immaterial)? Or is there no need to ‘fix the system’?

This has various parts to it, but as a journal editor for the past five years I actually disagree with the first part.  It has always been difficult to get reviewers, and I read plenty on Twitter, Facebook, Political Science Job Rumors, etc. from people complaining about how many review requests they receive.

That brings me to the APSR.  I was really struck by, and very much identify with this note from the editors in the February 2011 issue.

On a less happy note, we continue to experience some very late or ultimately non-responsive referees. We urge those referees who are not sure they can complete the review in time to decline, rather than accept and then run far over deadline-or, if unforeseen circumstances arise, to let us know so that we may assign another referee. And we welcome authors' inquiries about delay, which often spur us (and dilatory referees) into action.

People who cannot be bothered to finish reviews should never accept them in the first place.  Authors complain about the length of time the review process takes, but very often this is the result of waiting (while also bugging the authors) for late reviews or simply waiting for potential reviewers to respond in the first place.  The fact that the flagship journal in political science suffers from this shows how pervasive this is (whether it is more pervasive than in the past is open to debate).

In short, we want journals to complete the review process quickly and publish our scholarship but at the same time we screw up that process for everyone else.  I can easily imagine someone missing a deadline for a review while simultaneously emailing an editor to ask what is taking so long.

This brings me to the question of incentives mentioned in the Crooked Timber post.  Unfortunately, I cannot think of any.  It would be nice if reviewers could be paid, as they are for reviewing full-length book manuscripts, but that is obviously impossible.  Ideally, quality reviews could be submitted with other materials as part of the merit process in individual departments.  Even that, though, could only go so far.

Is the process sustainable?  Yes, there is no doubt in my mind that it is.  If two things happened, the process could be improved considerably, and neither is difficult.

First, editors need to search harder for reviewers.  Yes, it is easiest to either do a quick Google Scholar search or take a look at the TOCs of recent top journals but these tend to generate the same names over and over.  There are lots of very qualified reviewers that are not burned out and some, in fact, even email me and ask that I keep their name in mind when an article on X topic is submitted.

Second, if you are asked to do a review and cannot, then always make it a point to suggest at least one other name to the editor.  Further, for those reviewers with Ph.D. programs, suggest your advanced graduate students.  Almost without fail, those reviewers are deeply immersed in the literature and conscientious, which leads to high quality reviews.  Even just one name to the editor speeds up the process a lot.