Now also cross-posted at homolog.us (and slightly edited here to remove any potentially misleading, unintentional implications).

There is a lively discussion going on right now in various forums on the incentives for scientists to publish their work in this venue or another. Some of these discussions cite our manuscript on the pernicious consequences of journal rank, others don't. In our manuscript, we speculate that the scientific community may be facing a deluge of fraud and misconduct, because of the incentives to publish in high-ranking journals, a central point of contention in the discussions linked to above.

However, one need not go to the extreme and (still) very rare cases of misconduct. The pernicious incentives in our reputation system can also lead to much more subtle behaviors that appear rather inocuous at first. For instance, in order to market our research to the journal Science, we invented the term "operant reward learning". This term does not exist, but we felt that the term 'reward' was a buzz-word that increased our chances - and it did get published. If everyone did that, it's easy to imagine what a mess (even worse than it already is) the scientific nomenclature would be. Another example of just how subtle these incentives may skew the scientific debate happened to land on my desk this morning in the form of a paper quite close to my own field of research, published in the (for our field) very highly ranked journal "Current Biology".

This paper caught my attention not only because it concerned fly behavior or featured a colleague I happen to know quite well, but because it stated in the abstract that: "blocking synaptic output from octopamine neurons inverts the valence assigned to CO₂ and elicits an aversive response in flight". We currently have a few projects in our lab that target these octopamine neurons, so this was a potentially very important finding. It was my postdoc, Julien Colomb, who spotted the problem with this statement first. In fact, if it wasn't for Julien, I might have never looked at the data myself, as I know the technique and I know and trust the lab the paper was from. I probably would just have laid the contents of the abstract to my memory and cited the paper where appropriate, as the results confirmed our data and those in the literature (a clear case of confirmation bias on my part). But have a look at Fig. 3 (click for larger image):



The important data to look at is in Fig. 3D. It shows an attraction for CO₂ vs. air in wildtype flies (WT), but an aversion in the genetically manipulated flies (Tdc2-GAL4/UAS-TNT). This is what is stated in the abstract: wildtype flies are attracted to CO₂ and flies where octopamine release is blocked, avoid CO₂. However, the important control experiments, are those that test for off-target effects of the genetic manipulation. In other words, do the transgenes inserted into the fly genome have an effect of their own, independent of their combined effect on octopamine? In this case, there are two transgenes, a GAL4 transgene (the driver) and a UAS transgene (the effector). Their CO₂ scores are shown at the end (Tdc2-GAL4/+ and UAS-TNT/+, respectively). Interestingly, these lines both show a strongly reduced preference for CO₂. Their preference is so strongly reduced that it is not even different from that for air. To put it differently: neither of both control lines show normal, wild type behavior. They may not be able to detect CO₂ any more, or have secondary alterations that simply reduce the preference for CO₂, or a myriad of other explanations. Importantly, nobody can know if these two effects, which either alone already reduce the preference for CO₂ dramatically, together could lead to an avoidance of CO₂ that is completely independent of the targeted octopamine neurons.

In this respect it is important to point out that the authors are not trying to hide this effect. In the text, in what appears to be a contradiction to the abstract, the authors write: The last sentence, of course, is incorrect: if both lines independently reduce the attractiveness of CO₂, then it is very conceivable, one might even say straightforward, that both together might reduce it so much, that the resulting value of CO₂ is negative, leading to an aversive response in the flies, irrespective of the involvement of octopamine.

Given what is known about the action of octopamine in these processes, the hypotheses that the authors claim to have corroborated is beautiful, makes sense and is biologically plausible. So the result they present in the abstract "blocking synaptic output from octopamine neurons inverts the valence assigned to CO₂" makes this a very sexy paper for the field that unites several disparate findings and puts a whole set of results in a broader perspective (and may well be correct!). Of course, these considerations are crucial for marketing your paper to one of the top journals in the field. Had the authors discarded the octopamine results from their paper, one may speculate that it would be rather unlikely it would have been published in Current Biology. It is more difficult to estimate what might have hapened if the authors had been more conservative in their approach and rephrased the statement in the abstract to something that would indicate that they had suggestive, but not conclusive evidence for the involvement of octopamine neurons in CO₂ preference. A reasonable speculation would be that reviewers might have asked for additonal experiments until such a conclusion could be reached.

To make this unambiguously clear: I can't find any misconduct whatsoever in this paper, only clever marketing of the sort that occurs in almost every 'top-journal' paper these days and is definitely common practice. On the contrary, this is exactly the behavior incentivized by the current system, it's what the system demands, so this is what we get. It's precisely this kind of marketing we refer to in our manuscript, that is selected for in the current evolution of the scientific community. If you don't do it, you'll end up unemployed. It's what we do to stay alive.

In this respect it is worth speculating about the particular incentives the authors of this study might have experienced. The first author is a postdoc in Mark Frye's lab, so she needs to publish in top journals to get a job. The second author was an undergraduate, so likely less involved in the drafting and revising of the paper and the last author is a junior investigator for HHMI, so likely under enormous pressure (or at least perceived pressure) to publish in top journals not only to justify his award, but also to do well in future evaluations. Note that these are pure speculations: while I know Mark Frye personally, I did not contact him or any of the authors for a comment, as I felt the paper should be appraised on its own.

Obviously, this is just a case study, N=1, an anecdote, but I think it exemplifies the incentives and how they can distort the scientific debate. For instance, see the Tweet I sent around after I read the abstract (but before I had a look at the actual data):

Octopamine reverses Carbon Dioxide preference in flies feedly.com/k/Upirw7— Björn Brembs (@brembs) January 25, 2013

UPDATE: Due to the popularity of this post, I'd like to spell out what I alluded to above: there would have been nothing wrong with the paper, had the abstract mentioned that the connection with octopamine was suggestive, but not conclusive.

---

Wasserman, S., Salomon, A., & Frye, M. (2013). Drosophila Tracks Carbon Dioxide in Flight Current Biology DOI: 10.1016/j.cub.2012.12.038