3 ms·

It is so incredibly easy to fabricate data that looks real and which has provenance going back to "raw data files" or whatever - Just generate random values wit

It is so incredibly easy to fabricate data that looks real and which has provenance going back to "raw data files" or whatever - Just generate random values with the statistical properties of other published data on similar topics. A little more effort if there's cryptographic signing of files (put fake samples into the machine instead, no big deal that's what you do to calibrate the machine in the first place) or hand written questionnaires (actually this could be tricky without an army of complicit grad students). So it's shocking when successful high profile researchers are this sloppy and manage to get away with it for so long.

rossdavidh 3y ago
In fact, it strongly suggests that fabricating data is so common, that it doesn't seem dangerous or risky to them, and they take no great amount of care while doing it. Familiarity breeds contempt. I'm pretty sure the researchers implicated are smart enough to figure out how to fabricate data well, if they wanted to; that they did not bother to do so, suggests a cultural rot.
- tnecniv 3y ago
  Familiarity for the researcher breeds contempt. If you succeed once and don’t get caught, it’s easier to do it the second time. It doesn’t require a culture of dishonestly for bad actors to get comfortable breaking rules.
  - rossdavidh 3y ago
    You're correct that I stated that badly. I didn't mean cultural rot as in "everyone does it", I meant cultural rot as in there isn't much perceived chance of being caught, i.e. not much of a culture of double-checking each others' work, so they do it again and again (until they get sloppy). But you're correct that it doesn't mean most people are doing it.
juujian 3y ago
It's a real problem. The whole issue complex has turned me off of using regression analysis in my work completely. If there is data, I just show the data as good as possible. But as long as regression analysis is a useful tool to advance your career (as opposed to research) this will continue.
- tnecniv 3y ago
  There’s no reason you can’t do both. Summarizing Tufte: you should always show the data if possible, but you can both show the data and analysis of the data. Don’t throw the baby out with the bath water.

rossdavidh 3y ago

In fact, it strongly suggests that fabricating data is so common, that it doesn't seem dangerous or risky to them, and they take no great amount of care while doing it. Familiarity breeds contempt. I'm pretty sure the researchers implicated are smart enough to figure out how to fabricate data well, if they wanted to; that they did not bother to do so, suggests a cultural rot.

tnecniv 3y ago
Familiarity for the researcher breeds contempt. If you succeed once and don’t get caught, it’s easier to do it the second time. It doesn’t require a culture of dishonestly for bad actors to get comfortable breaking rules.
- rossdavidh 3y ago
  You're correct that I stated that badly. I didn't mean cultural rot as in "everyone does it", I meant cultural rot as in there isn't much perceived chance of being caught, i.e. not much of a culture of double-checking each others' work, so they do it again and again (until they get sloppy). But you're correct that it doesn't mean most people are doing it.

juujian 3y ago

It's a real problem. The whole issue complex has turned me off of using regression analysis in my work completely. If there is data, I just show the data as good as possible. But as long as regression analysis is a useful tool to advance your career (as opposed to research) this will continue.

tnecniv 3y ago
There’s no reason you can’t do both. Summarizing Tufte: you should always show the data if possible, but you can both show the data and analysis of the data. Don’t throw the baby out with the bath water.