5 ms·

Whether or not your papers pass peer review—-and which journals it is published in—are important criteria for hiring, tenure, whether your grants are funded, et

• bumby 3y ago
  >If you get rid of peer review, it’s not science. It’s just a vanity press. If you believe replicability is central to science, the current paradigm doesn't necessarily converge on science either. And when people are graded on how many publications they garner, it borders on turning publication into a symbol of status rather than one of science.
  • rhelz 3y ago
    > If you believe replicability is central to science. I do believe that, but it doesn’t matter what anybody believes, replicatable experiments and results, which peers can review and agree on, are the soul of science. Without that it’s not science, it’s just creative writing.
    • bumby 3y ago
      Then don’t you think the first part of that process is explaining the methods and results of your experiment? That’s precisely what the current situation does. What’s lacking is the replication incentives.
      • rhelz 3y ago
        I propose to supply those incentives—-pay for good reviewer and replicators. If you want something done, you gotta pay for it. We can’t just rely on volunteers.
        • bumby 3y ago
          I'm on board with that idea, as long as we can also provide guardrails against the perverse incentives of paying for them. E.g., we need to avoid frivolous reviews/replication as well as something evolving into a "pay for a good review" service.
          • rhelz 3y ago
            Well, if you are paying somebody to do something, you gain a lot of leverage: 1. You can negotiate a due date. No more waiting for years before the journal's reviewers actually review your paper. 2. You can negotiate a set of deliverables. You can specify that they can't just say "this sux"; they have to show the lines where the big hairy proof are wrong, or if its an algorithm, they have to actually code it up and run it themselves before they say it doesn't work. 3. You can more reliably attract good reviewers. If you aren't begging for people to volunteer, but you are paying good money, you can be a lot pickier about who you hire. I mean, I've been a consultant: what are the guardrails that I won't rip-off my clients? I don't want to ruin my reputation, I want repeat business, and I want to be able to charge a high hourly rate because I deliver a premium product. Same guardrails would apply to peer reviewers and to reproducers.
            • bumby 3y ago
              Sure, but I’m poking at the bad leverage you can also wield. 1) you can create undue schedule pressure that results in a cursory review that may not catch the more nuanced problems in your investigation. 2) you can be more belligerent about not sharing data. If they want to get, they won’t argue. 3) you can pay for reviewers who you know will give a positive review. Without guards against this, it’s almost a certainty that the glut of PhDs will result in some treating it like a scam de hustle where it’s more about the economics than the science. Some consultants are well known to play the game where they tell clients what they want to hear rather than what they need to hear. I don’t think consultancy is a good model for this.
              • rhelz 3y ago
• geysersam 3y ago
  And the current system is a holy grail that can never change? You can still based your hiring based on reviews made by peers