• Tweet

Now that the preliminary NIPS reviews have been out, it is time to remember that a rejected paper is not the end. In particular if you’re a beginning graduate student and you’ve submitted your first paper, be advised that you need to keep on working on your paper, and the worst that can happen is if you let yourself be discouraged and just discard the hours of work you’ve put into your paper by stopping to work on it.

First, you need to understand that there are many different reasons why your paper was rejected.

• NIPS reviewers are usually heavily overloaded. A NIPS reviewer is typically asked to review about eight papers. Given the already high workload, this means that you end up reviewing eight papers in two to three days. So if your paper is less than clear, or the idea doesn’t easily fit in one of the existing frameworks, chances are high that the reviewer does not “get it”, and plays it safe by rejecting your paper.

• Often, reviewers offload some of the papers to their graduate students and Post-Docs which are sometimes not yet experienced enough adopt a high-level view of your paper and to give a faithful evaluation of the merits of your paper.

• At a competitive conference such as NIPS, one lukewarm review is enough to get your paper rejected. So a single reviewer can get your paper rejected.

• Finally, chances are high that while your approach and idea have sufficient merit, you failed on some of the other aspects. For example, you haven’t sufficiently put your work into the context of the existing body of work, or your experimental evaluation was faulty.

Almost no paper is accepted in its first submission. Everyone has some war stories about papers which were accepted only after several submissions, after switching journals, complete rewrites and appealing to the action editor. Each reject, even if the reviews are completely off the mark, is an important feedback on how your work is perceived and gives you hints on how to improve your paper.

In summary, a reject is not the end, in fact, it is only the beginning:

• If possible, you can write a rebuttal, clearing up misunderstandings. In my experience, however, this has seldom led to an improvement of scores.

• Resubmit an improved version to another conference. The advantage of a conference is a somewhat predictable review schedule. The disadvantage is that you will usually get new reviewers with the danger that one of them will again not like your paper.

• Resubmit your paper to a workshop. Workshops usually have much lower thresholds and also don’t lead to a “real” publication, but they allow you to present your work to others and get feedback, and someone in your audience might be your next reviewer.

• Resubmit to a journal. This might require to improve your paper a lot. The main advantage of journals is that the review process has several rounds. After a reject, you typically are assigned to the same reviewers, meaning that you can actually work with the reviewer to improve your paper to the point where it will be accepted.

Posted by Mikio L. Braun at 2010-08-03 10:20:00 +0200