5 ms·

> replicating a study done in complete seriousness and honesty is hard and is not guaranteed to yield the same result Yes, a study failing to replicate is not

> replicating a study done in complete seriousness and honesty is hard and is not guaranteed to yield the same result Yes, a study failing to replicate is not in itself evidence of fraud. Nor is it definitive evidence that the effect shown in the original doesn't exist, since a replication can be messed up too. But science needs replication because we're ultimately trying to learn about the world, and an effect we can't reliably produce is usually an effect we're not sure exists. Replication sometimes catching dishonesty is a bonus: failure to replicate calls the effect into question, which calls the method used to find that effect into question, which sometimes calls into question whether that method was actually used. (Or in the case of bad statistics, whether it was chosen to dishonestly show a certain result.) It's the appropriate propagation of doubt.

kergonath 5y ago
> Yes, a study failing to replicate is not in itself evidence of fraud. Nor is it definitive evidence that the effect shown in the original doesn't exist, since a replication can be messed up too. It’s not a trial to determine the guilt of a supposed fraudster. This is what institutions should do because fraud is a breach of contract (in all sane institutions). However, the main problem is made up results in the scientific record. This is what we attempt to solve by retracting articles. And from the point of view of the scientific record, there is no meaningful difference between a made up result and something that cannot be replicated.
- joe_the_user 5y ago
  This is what we attempt to solve by retracting articles. And from the point of view of the scientific record, there is no meaningful difference between a made up result and something that cannot be replicated. I don't think "the scientific record" is more than a metaphor that's referred to in talking of retraction. Still, if one were to imagine a hypothetical scientific record, you would have to imagine something that's inherently malleable. Even if most basic results are never going to be disproved, so fraction of even the most basic result are going to be disproved. The thing, it's unusual but not unheard-of for a researcher to come up with a result, have it not be replicated, have the result languish and then have a later researcher take advantage of the result, either show it could be replicated or otherwise explaining what was actually going on. But if the explanation for a strange result is "they faked it", then the impact of the situation is different. Moreover, non-reproducibility is a reason for retraction but I don't think it's the main reason and well know papers that haven't reproduced aren't necessarily retracted Afaik. Which is to say, I think there's still an important difference between non-reproducible for unknown reasons and know-to-be-fabricated data. Article on retraction practices and "correcting the scientific record" https://pubs.acs.org/doi/10.1021/acs.chemmater.9b00897 https://pubs.acs.org/doi/10.1021/acs.chemmater.9b00897

kergonath 5y ago

> Yes, a study failing to replicate is not in itself evidence of fraud. Nor is it definitive evidence that the effect shown in the original doesn't exist, since a replication can be messed up too. It’s not a trial to determine the guilt of a supposed fraudster. This is what institutions should do because fraud is a breach of contract (in all sane institutions). However, the main problem is made up results in the scientific record. This is what we attempt to solve by retracting articles. And from the point of view of the scientific record, there is no meaningful difference between a made up result and something that cannot be replicated.

joe_the_user 5y ago
This is what we attempt to solve by retracting articles. And from the point of view of the scientific record, there is no meaningful difference between a made up result and something that cannot be replicated. I don't think "the scientific record" is more than a metaphor that's referred to in talking of retraction. Still, if one were to imagine a hypothetical scientific record, you would have to imagine something that's inherently malleable. Even if most basic results are never going to be disproved, so fraction of even the most basic result are going to be disproved. The thing, it's unusual but not unheard-of for a researcher to come up with a result, have it not be replicated, have the result languish and then have a later researcher take advantage of the result, either show it could be replicated or otherwise explaining what was actually going on. But if the explanation for a strange result is "they faked it", then the impact of the situation is different. Moreover, non-reproducibility is a reason for retraction but I don't think it's the main reason and well know papers that haven't reproduced aren't necessarily retracted Afaik. Which is to say, I think there's still an important difference between non-reproducible for unknown reasons and know-to-be-fabricated data. Article on retraction practices and "correcting the scientific record" https://pubs.acs.org/doi/10.1021/acs.chemmater.9b00897 https://pubs.acs.org/doi/10.1021/acs.chemmater.9b00897