linking back to brembs.net






My lab:
lab.png
The most interesting presentation at the Berlin Open Access Days yesterday was the one from Max Planck geochemist Ulrich Pöschl. He was telling us about the success of his open access journal Atmospheric Chemistry and Physics. Weirdly enough, just as I wanted to look one of Pöschl's publications up, it appeared magically on FriendFeed.
He started out by stating some of the problems facing the scientific community in an age of seemingly ever increasing numbers of scientific publications. He felt that the two major problems were the need to somehow filter this information to only get at the "best" science and the problem that many papers are so sloppily written, that it takes another lab much too long until it can reproduce the already published data. The first problem results in what he aptly called the "dilution of information" whereas the second problem leads to waste and misallocation of rsoures. While I would have offered quite different solutions to these problems than he presented (a single scientific database with a combination of social networking, post-publication review spiced up by a reputation system for the filter problem and JoVE for the second problem), I was neverheless rally impressed and intrigued by what he had to say about the interactive peer-review system he hs establshed in his journal. You can find some older slides whch are fairly similar to his presentation here (PDF), but I'll outline his talk so you get the gist.
Obviously, editors and referees have only limited capacity for review and as such more submissions inevitably means slower publication times, i.e. papers get stuck publication traffic. Often, very important, critical reviews in closed review systems never see the light of day (beyond authors and editors, that is), further slowing down progress. Traditional commenting on published articles used to catch some of these critical points, but it has dropped from about 1/20 in 1978 (comments/paper) to only about 1/100 in 1998. The conflict is clear: science needs rapid publication but must not give up on thorough peer review. How can this be acomplished? I've taken the liberty of copying one of his graphs from his presentation to explain the system he has established (click to enlarge):

acp_small.jpg

First, the initially submitted mansucript is handled by an editor who picks the reviewers. The manuscript gets published as a "discussion paper" and both the reviewers and any reader can comment on it, either anonymously or openly. This review period lasts about 8 weeks and can of course be repeated for several iterations until the editor finds all concerns addressed. Once the editor accepts the discussion paper for publication, it gets published in its final form in a different journal, at the moment without the possibility of any further commenting directly on the paper (traditional commenting, of course, is still possible).
This transparent form of interactive review provides a huge incentive foreveryone for a constructive discussion: author don't want to embarrass themselves publicly and thus are very careful in drafting their manuscripts. Reviewers and readers still have influence on the final version of the paper and don't want to risk this influence by using personal attacks or unwarranted criticisms in public. This system works great in preventing many drawbacks of traditional peer-review such as hidden obstruction and plagiarism, and at the same time provides authors with the means of rapidly publishing their research results. All critical comments on the research are kept and public. I agree with Pöschl hat widesprad adoption of this system would also help reduce the flood of papers by deterring careless, useless and false papers.
ACP manages to publish about 400 papers/year with ~70 editors and a total cost of each paper for the authors of about 1000€ (~1400US$). They have about 10.000 comments so far and had only to delete two of them for being inappropriate. This makes about 5 comments per discussion paper where ~3-4 are from authors and reviewers. Pöschl told me afterwards that he thinks the reason why their commenting system works so well is because the potential of having an impact on the form of the final paper is a strong motivating factor in the whole commenting game. I thought this was a very convincing argument. They managed to reach the top impact factor of their field with a rejection rate of only about 10-20%. The total time from initial submission to final paper is 3-6 months with only about 1 month to initial publication as discussion paper. I don't remembr if he showed the final slide of the PDF file I linked above, but I think I would've because all his points are right down my alley grin.png:
Promote open access publishing
  • prescribe open accessto publicly funded research results
  • transfer fundsfrom subscription to open access publications:
    convert subscription budgets (e.g. 10-30 % per year) into OA publishing funds (e.g., 2000 EUR per year & scientist, plus project-specific funds)
Emphasize quality assurance & interactivity
  • foster open access publishing & collaborative peer review:
    implement discussion forums in new & existing journals
  • mere access is not enough(repositories & self-archiving)
Improve scientific evaluation & rating methods
  • evaluate individual papers not just journal impact factors
  • refine statistical parameters for citation, download, and usage;
    interactive commenting & rating
IMHO, this is really something the folks at PLoS One should look at to try and see if there's anything they can take home from this system to implement in PLoS One. I thought this all made a lot of sense.


Posted on Friday 10 October 2008 - 17:15:19 comment: 0
{TAGS}


You must be logged in to make comments on this site - please log in, or if you are not registered click here to signup
Render time: 0.0922 sec, 0.0066 of that for queries.