3 ms·

No. Absolutely not. There is no non-fraudulent explanation here. In the best case of "poorly described methods", we're talking about authors who blatantly lied

No. Absolutely not. There is no non-fraudulent explanation here. In the best case of "poorly described methods", we're talking about authors who blatantly lied about how they sampled their patients and deliberately omitted what would have been an extremely laborious process, involving many thousands of extremely non-consecutive patients, to produce an extremely well-matched treatment and control group. You've got your choice as to the exact nature of the enormous lie or fabrication, but there's no possibility of innocent omission.

radford-neal 5y ago
I think it's difficult to eliminate the possibility of a tremendous degree of incompetence, without outright fraud, based just on the p-values near 1. As noted in other comments here, one possibility is that the authors failed to correctly describe the procedure they used. Sheldrick does consider this, pointing out that even if they actually did try to match subjects, unlike what they said in the paper, it would be very unlikely that they could match them well enough to obtain the reported p-values. But another possibility is that they were totally incompetent at using their statistical software - that the reported p-values are not actually the real p-values for the tests that they said they did. Maybe they accidently used the wrong dataset, for example. Or perhaps they copied numbers from the wrong column of the table the software produced. In a wider sense, though, this would still essentially be fraud - it's dishonest to publish a paper that purports to be a competent study when really you are totally clueless about how to do research, or care so little about correctness that you put no effort into checking that you did things correctly. (And surely they would have to suspect that they don't really know what they're doing...)
- civilized 5y ago
  This isn't about reported p-values. Sheldrick computed p-values from the reported treatment and control group samples. The latter are basic count data, not the output of statistical software. So here we are talking about either the raw data being wrong, or this tiny amount of raw data being incorrectly summarized. Something like a column duplication would have resulted in perfectly identical data between treatment and control groups, which this data does not show. The best case scenario is an errant copy-paste or Excel formula that somehow presented both groups of data in a way that actually only looked at one of the two groups, but also somehow introduced small differences along the way. How exactly this could happen is beyond my imagination, but perhaps it introduces a sliver of doubt as to whether the paper is fraud by intention or merely by negligence.

radford-neal 5y ago

I think it's difficult to eliminate the possibility of a tremendous degree of incompetence, without outright fraud, based just on the p-values near 1. As noted in other comments here, one possibility is that the authors failed to correctly describe the procedure they used. Sheldrick does consider this, pointing out that even if they actually did try to match subjects, unlike what they said in the paper, it would be very unlikely that they could match them well enough to obtain the reported p-values. But another possibility is that they were totally incompetent at using their statistical software - that the reported p-values are not actually the real p-values for the tests that they said they did. Maybe they accidently used the wrong dataset, for example. Or perhaps they copied numbers from the wrong column of the table the software produced. In a wider sense, though, this would still essentially be fraud - it's dishonest to publish a paper that purports to be a competent study when really you are totally clueless about how to do research, or care so little about correctness that you put no effort into checking that you did things correctly. (And surely they would have to suspect that they don't really know what they're doing...)

civilized 5y ago
This isn't about reported p-values. Sheldrick computed p-values from the reported treatment and control group samples. The latter are basic count data, not the output of statistical software. So here we are talking about either the raw data being wrong, or this tiny amount of raw data being incorrectly summarized. Something like a column duplication would have resulted in perfectly identical data between treatment and control groups, which this data does not show. The best case scenario is an errant copy-paste or Excel formula that somehow presented both groups of data in a way that actually only looked at one of the two groups, but also somehow introduced small differences along the way. How exactly this could happen is beyond my imagination, but perhaps it introduces a sliver of doubt as to whether the paper is fraud by intention or merely by negligence.