Last year, John O'Doherty, Rui Costa and I applied for one of the research conferences of the European Science Foundation. All three of us work on the neurobiology of spontaneous actions, so the topic of the call - "Action" was perfect for our research focus: The idea was to bring people together who had helped bring about the current paradigm shift in the neurosciences, reviewed e.g. here, here or here, but who hadn't necessarily met before or talked to each other. As the evidence helping to change the perspective on general brain function had come from many different disciplines and has ramifications for aspects of society outside of science, we had a very broad and integrative line-up of speakers and participants (as you can see in our application), from neuroscience, ecology and medicine to art and philosophy. Moreover, all three of us had just returned from stays in the USA and were enthusiastically looking forward to organizing a conference here in Europe to broadcast what we had learned to European students and colleagues.

Our enthusiasm for European conferences has now cooled off, somewhat.

Last week, we got the reviews on our application back and two of them were very much in support of our application. The third one, however, cannot even be considered a review. Needless to say, our application was not funded. Basically, Reviewer #2 exclaimed that he/she didn't understand what our idea and purpose was and scored our application as "average or less" without any specific criticism. Here the first sentence of the 'review': There are two points here, which under normal circumstances would disqualify the reviewer. First, the reviewer apparently does not realize that there are brains other than human brains, or he/she would not have emphasized our use of the plural 'brains'. Lack of such basic biological knowledge would be an exclusion criterion in my book; apparently not for the ESF. Second, the proposers as well as the proposed speakers have contributed a substantial number of peer-reviewed publications providing scientific evidence giving rise to the change in perspective on general brain function. Since Reviewer #2 denotes this recent development as 'epistemological assumption', he/she is apparently not aware of the scientific literature and of the current trends in it. In my books, this is an exclusion criterion for a reviewer as well; apparently not for the ESF.

In the second and third sentence, the reviewer acknowledges his/her unsuitability already displayed in the first sentence: One may summarize the first three sentences of this 'review' as "I lack the basic background in this field and consequently I don't understand what the proposers are talking about". That the reviewer nevertheless goes on to review our proposal after this admission, can only be understood as a real-life example of the Dunning-Kruger Effect.

I also work as an editor in several international peer-reviewed journals. In this position, it is my task to always try and find the most competent reviewers. Sometimes, however, if I don't know the reviewer personally, it happens that the person is not quite as suited as it seemed from their publication record. The three sentences quoted above would be a dead giveaway that I had made a mistake in choosing this reviewer. My mistake would be grounds enough to either not forward this review to the authors and instead search for a replacement, or at least instruct the authors which portions of the review to disregard. Apparently, ESF peer-review practice does not include actually reading the reviews before they are assessed by the decision-makers or forwarded to the proposers after the decision has been made.

One final tidbit of the 'review' is worth quoting: Here again, the reviewer demonstrates lack of experttise. Without a proper understanding of the subject matter of our research, the reviewer has to resort to point out where our research is published rather then what we have published. It is telling in this regard that the other two reviewers did not have to mention where our research was published, in order to characterize our standing within the research community.

Open peer-review is already performed at a number of scientific journals such as Frontiers in Neuroscience. Also in PLoS One there is the option (and indeed the general practice) of publishing the peer-review comments with the final version of the paper. In my experience, this tends to reduce the number of unusable reviews. It is in the spirit of open peer-review that I'm publishing the full reviews here, along with our proposal, so anyone can form their own opinion.

Finally, it needs to be pointed out that while the remaining two reviewers did not give us the highest scores either (4 out of 5), at least they were competent and had excellent suggestions for our program which we certainly would have implemented, had it been funded. Thus, it is not the rejection itdelf which prompted this post, but rather the sub-standard peer-review procedure which is unfitting for an international funding institution such as the ESF. Of course, before posting the reviews here, I did contact the ESF for a comment. They replied with a short note thanking us for our feedback helping them to improve their peer-review procedures. I would like to see this post as a further incentive towards improving these procedures at the ESF and maybe get one or the other official to start thinking about implementing open peer-review options there.