5 ms·

Then how do you trust the replication-studies? Just keep replicating them, too? The overall process assumes trustworthiness. The better that assumption holds

by _Nat_ 5y ago

Then how do you trust the replication-studies? Just keep replicating them, too? The overall process assumes trustworthiness. The better that assumption holds, the better science performs.

EricE 5y ago
I don't think many serious scientists would consider assumptions a valuable part of the scientific method.
- _Nat_ 5y ago
  There wouldn't be a point to reading papers/textbooks, buying equipment from others, etc., if you thought that it was all part of some elaborate deception. The general presumption would tend to be that other scientists are trying to do good, honest work. Scientific progress does better when that presumption turns out to be more true. To be clear, I was responding to their apparent disregard for the harm done by the original alleged-fraud.
rmbyrro 5y ago
It's not a flawless system. Science is highly reliant on humans and we're not reliable. But it's reasonable to think that some reproduction is better than no reproduction.
- _Nat_ 5y ago
  Checks are good, definitely. And they help to limit the damage caused by stuff like fraud. You were right about that stuff. The point I think folks were contesting was the tone some may've inferred from [this comment](https://news.ycombinator.com/item?id=30787447 https://news.ycombinator.com/item?id=30787447 ), which may sound a bit like downplaying the damage caused by fraud under the premise that reproduction ought to catch such things anyway. For a computing analogy, TCP uses mechanisms that enable it to tolerate packet-loss. But packet-loss still slows things down, limiting bandwidth and increasing latency, until a connection would be impractical. For a connection with many hops, even a small packet-loss-rate along each hop would be a huge problem, even though TCP actively checks for packet-loss and tries to fix it. Likewise, science-with-replication presumably ought to be able to fix a lot of fraud and errors -- but those problems take a heavy, debilitating toll on scientific progress; it's not a small problem that replication-studies alone could satisfactorily rectify.
  - rmbyrro 5y ago
    What the people inferring from my comment are missing is that the reproduction efforts actually saved us from damage. The real damage would have been everyone believing the paper conclusions were solid and start to act according to this false belief. Reproduction efforts saved us from that damage. These efforts were not the damage. Whoever is seeing differently doesn't understand one of the basic foundations of how the scientific endeavor works. And again: anyone expecting a world where there are no dishonest scientists or where the dishonest ones call themselves out in advance to save from the "damage" of people spending time reproducing their papers, is just playing wishful thinking, which is not a recipe for success...
    - _Nat_ 5y ago
      > Reproduction efforts saved us from that damage. [...] These efforts were not the damage. I'm pretty sure everyone firmly agrees with this.

by _Nat_ 5y ago

Then how do you trust the replication-studies? Just keep replicating them, too? The overall process assumes trustworthiness. The better that assumption holds, the better science performs.

EricE 5y ago

I don't think many serious scientists would consider assumptions a valuable part of the scientific method.

_Nat_ 5y ago
There wouldn't be a point to reading papers/textbooks, buying equipment from others, etc., if you thought that it was all part of some elaborate deception. The general presumption would tend to be that other scientists are trying to do good, honest work. Scientific progress does better when that presumption turns out to be more true. To be clear, I was responding to their apparent disregard for the harm done by the original alleged-fraud.

rmbyrro 5y ago

It's not a flawless system. Science is highly reliant on humans and we're not reliable. But it's reasonable to think that some reproduction is better than no reproduction.

_Nat_ 5y ago
Checks are good, definitely. And they help to limit the damage caused by stuff like fraud. You were right about that stuff. The point I think folks were contesting was the tone some may've inferred from [this comment](https://news.ycombinator.com/item?id=30787447 https://news.ycombinator.com/item?id=30787447 ), which may sound a bit like downplaying the damage caused by fraud under the premise that reproduction ought to catch such things anyway. For a computing analogy, TCP uses mechanisms that enable it to tolerate packet-loss. But packet-loss still slows things down, limiting bandwidth and increasing latency, until a connection would be impractical. For a connection with many hops, even a small packet-loss-rate along each hop would be a huge problem, even though TCP actively checks for packet-loss and tries to fix it. Likewise, science-with-replication presumably ought to be able to fix a lot of fraud and errors -- but those problems take a heavy, debilitating toll on scientific progress; it's not a small problem that replication-studies alone could satisfactorily rectify.
- rmbyrro 5y ago
  What the people inferring from my comment are missing is that the reproduction efforts actually saved us from damage. The real damage would have been everyone believing the paper conclusions were solid and start to act according to this false belief. Reproduction efforts saved us from that damage. These efforts were not the damage. Whoever is seeing differently doesn't understand one of the basic foundations of how the scientific endeavor works. And again: anyone expecting a world where there are no dishonest scientists or where the dishonest ones call themselves out in advance to save from the "damage" of people spending time reproducing their papers, is just playing wishful thinking, which is not a recipe for success...
  - _Nat_ 5y ago
    > Reproduction efforts saved us from that damage. [...] These efforts were not the damage. I'm pretty sure everyone firmly agrees with this.