5 ms·

With anonymous publications special interest groups can drop an unlimited number of publications without exposing their network of shills. So while the proposa

• nickff 11y ago
  If the 'shills' are wrong, then the peer review should take care of them. The whole problem that this proposal is trying to eliminate is a combined problem of discrimination (for various reasons) and cargo-cult/groupthink; calling someone a 'shill' is an example of at least the first of these issues. If the peer review process cannot successfully refute claims in a paper, and has to resort to name-calling, that is a sign that the reviewers are either incompetent or lazy (, and that there may be fundamental flaws in the peer review system).
  • Alex3917 11y ago
    > If the 'shills' are wrong, then the peer review should take care of them. That's not what peer review does. Peer review isn't investigative journalism, nor is it a criminal trial, nor is it designed to figure out if the results of the study are correct or incorrect.
    • nickff 11y ago
      Peer review is used as a way of determining whether the methodologies or underlying logic used in a study are flawed (in a provable way), and what the severity of the flaw(s) is/are. >"Peer review isn't... designed to figure out if the results of the study are correct or incorrect." There is no way of verifying the correctness of most studies and papers; they are simply examined for flaws. I do not see any reason why papers by 'good' authors should be examined less critically than those by 'shills'.
      • icegreentea 11y ago
        That's the point. If you publish anonymously then you can publish methodologies and analysis that is up to "gold standard" while completely falsifying/manipulating data. The current system mostly relies on reputation and professional ethics/pride to filter out that kind of behavior even before the publication stage. The cost to replicate experiments is typically high - even in our current system we cannot replicate to the degree that we want. One presumes that in an anonymous system, the problem would get even worse.
  • kwhitefoot 11y ago
    > and that there may be fundamental flaws in the peer review system There are. One problem is that there are too many papers to be reviewed, another is that the papers generally do not include complete datasets. Also, where large volumes of data are involved it is common to perform computational transformations on that data to extract something; the source code and build process are not always described in such a way as to make the executable reproducible. And those are just some of the problems with peer review, others include more human problems such as insufficient remuneration for the reviewer, reviewers who have too much at stake to give an unbiased review (whether too positive or too negative), too few reviewers with the necessary expertise, etc. Even with competent and committed reviewers the system is flawed; unfortunately there doesn't seem to be any alternative.
    • nickff 11y ago
      I agree with your description of other problems with the current system, and think that anonymizing authorship will serve to force them to be solved. Many popular journals currently look at the author's position and previous works to filter out the unpopular and/or unprestigious authors; anonymization would eliminate the bias against young, innovative, and/or unconventional scientists, and force the journals to improve their process.
    • vehementi 11y ago
      An anonymous science conclave could require that the data is complete & calculations/code presented unambiguously... unless that can only really be detected upon analysis during a serious peer review, rather than some surface level filtering.
  • an_ko 11y ago
    I think you meant s/peer review/replication study/g. Peer review aims to check that papers are publishable, not refute or affirm them by replicating their results; that's a separate process. I agree shills will eventually get sorted out though. The truth always wins in the end. Just not as fast as just peer review.
    • nickff 11y ago
      Replication should be done more often, and anonymization may serve to catalyze such an increase. Peer review is supposed to examine a paper for flaws, and do so fairly for all authors. This determination of which papers are well thought-out and documented is then supposed to decide which are publishable. I think you are also tacitly describing a pessimistic view (which I share) of the reality of peer review, that it does not succeed in these lofty aims, and that publishability is influenced by a number of extraneous factors.
  • abdullahkhalids 11y ago
    It's a question of resources. Once a shill has been identified editors can reject them at the editorial stage without even sending it to the peer reviewers. If funding sources are also not disclosed in the paper, then the peer review process will be unable to determine bias.
    • nickff 11y ago
      You seem to be describing a 'fruit of the poisonous tree' view of funding, and saying that studies funded by certain authors are right to be discounted.[1] For one thing, anonymous publication could serve to discourage partisan funding of studies, as the donor would no longer be able to verify the authorship of any given paper. The use of 'credibility' as a criteria for publication also creates large biases against novice scientists, or scientists from unusual backgrounds (scientific and otherwise). In addition, 'credible' scientists should never be given any benefit of the doubt; if someone depends on this, they are not doing good work. [1] https://en.wikipedia.org/wiki/Fruit_of_the_poisonous_tree https://en.wikipedia.org/wiki/Fruit_of_the_poisonous_tree
  • sliverstorm 11y ago
    The amount of energy required to refute bullshit is generally orders of magnitude less than that required to produce it.