In How to Publish a Scientific Comment in 1 2 3 Easy Steps you will read the crazy struggle the author had to go through to comment on a journal article that used incorrect data. It's sad and funny at the same time.
However, I think the best part of the article is at the end, which I include here in case you can't make it to the end of the article:
1. All data and parameters associated with any open publication should be available to anyone interested in it. The NIH has mandated this for its grant recipients, but sharing data and parameters should also be a required condition for a publication in any journal, no matter the funding source. Refusing to do so after a paper is published should be considered scientific misconduct.
2. Anyone knowingly publishing a paper that clearly contradicts the work of another group should be required, also as a condition for publication, to discuss the matter with that group well before publication. In the past, this was considered good scientific etiquette, but gone, apparently, are those days, so a rule is in order.
3. Journal editors should be more aware of referee conflicts of interest. Reviewers should be required to stipulate any conflict of interest in reviewing a paper, even if it's simply that they don't like the authors.
4. No journal editor should be allowed to edit a Comment on a paper that he allowed to be published. This is an obvious, unacceptable conflict of interest.
5. Comments should not be required to be so short as to prevent them from making sense. I suggest two journal pages, or, better, three. Or how about this radical idea: they should be as long as it takes to make the point.
6. Crazy rules that allow logically offensive situations, like the one that called for rejecting a Comment because the Reply is unpublishable, should be deleted immediately. And Comments and replies need not, and should not, be published together. The Comment should be published as soon as it is accepted, and the Reply as soon as it is. Indeed, a Comment on a Reply is also a good idea, yielding an interesting ongoing dialog that would benefit the community.
7. Reviewers who competently review a Comment should also review the Reply. They're the best qualified, as they're already familiar with the work. This would prevent the insane situation that occurred here, in which the highest quality review of the Comment was simply ignored.
8. Reviews should themselves be reviewable. Currently, reviewers can say whatever they like, and there is no check on them. Authors should be allowed to nominate
irresponsible reviewers, such as Reviewer #2 in the above scenario, whose review would be reviewed by another anonymous reviewer. Confirmed irresponsible reviewers should then be identified and removed from reviewer databases, which would be shared with other journals. Writing an irresponsible review should be considered a form of scientific misconduct.9. While removing unethical reviewers would help, improving reviews of ethical ones is also important. Currently there is no compensation of any sort for reviewers and hence no encouragement to do a good job. I believe that reviewers should be paid for their services. People take paid jobs much more seriously than volunteer efforts. Knowing this, social psychologists pay their subjects simply to fill out questionnaires because it yields much higher-quality results. And what could be more important than the accuracy of the archival scientific literature?
10. Require scientific ethics courses in grad school. Problems like those that I encountered are a proverbial ticking time bomb for science. What if those opposed to taking action against global warming were to make the claim that science shouldn't be believed in this matter because its process is so rife with poor ethics that it can't be trusted?
11. Finally, let's face it: most journal editors are simply too arrogant and have lost sight of the goal, which, it appears that I need to remind them of here, is to publish only truth. Perhaps they could be required to take a course or two in humility.
In light of the issues mentioned above, it really is time we move journals entirely online so that comments, replies, and data for a journal article can all be merged together. I'm optimistic of SSRN's current efforts to move into this arena.
via Greg Mankiw and Craig Newmark
