4 ms·

> they couldn't replicate it because they're idiots If they did not follow the steps to replicate (pre-training, using less compute, etc.) and then failed, so

• twothreeone 2y ago
  It's not a value judgement, just doesn't help his case at all. He'd need to counter the replication problem, but apparently that's not an option. Instead, he's making people who were unable to replicate it look bad, which actually strengthens their criticism.
  • skybrian 2y ago
    I don't know how you rebut a flawed paper without making its authors look bad? That would be a general-purpose argument against criticizing papers. Actually, people should criticize flawed papers. That's how science works! When you publish scientific papers, you should expect criticism if there's something that doesn't look right. The only way to avoid that is to get critical feedback before publishing the paper, and it's not always possible, so then the scientific debate happens in public.
    • twothreeone 2y ago
      The situation here is different though.. If I'm making an existence claim by demonstrating a constructive argument and then being criticized for it, the most effective response to that critique would be a second, alternative construction, not attacking the critic's argument. After all, I'm the one claiming existence.. the burden of proof is on me, not my critics.
      • skybrian 2y ago
        I don't know which argument is more constructive, though? Both teams reported what they did. They got different results. Figuring out why is the next step, and pointing out that they did different things seems useful. Though, the broader question is how useful the results of the original paper are to other people who might do the same thing.
  • xpe 2y ago
    > But saying "they couldn't replicate it because they're idiots, therefore it's replicable" is not a rebuttal, just bullying. > It's not a value judgement, just doesn't help his case at all. Calling it "bullying" looks like a value judgment to me. Am I missing something? To me, Dean's response is quite sensible, particularly given his claims the other papers made serious mistakes and have potential conflicts of interest.
    • twothreeone 2y ago
      I'm not saying "Bullying is bad and bullies are bad people", that would be a value judgement. I'm saying bullying is the strictly worse strategy for strengthening his paper's claims in this scenario. The better strategy would be to foster an environment in which people can easily replicate your claims.
      • xpe 2y ago
        I think for most people the word “bullying” has a value judgment built-in.
      • xpe 2y ago
        In a perfect world, making a paper easier to replicate has advantages, sure. (But it also has costs.) Second, even a healthy environment can be undermined by lack of skills or resources, intellectual dishonesty, or conflicts of interest.
      • xpe 2y ago
        Are you suggesting Dean take a different approach in his response? Are you saying it was already too late given the environment? (I’m also not sure I know what you mean by environment here.)
  • danielmarkbruce 2y ago
    they have open source code.