4 ms·

I bet that if the government goes after PIs who take federal money and then run a paper mill with inadequate supervision of postdocs, the situation vastly impro

I bet that if the government goes after PIs who take federal money and then run a paper mill with inadequate supervision of postdocs, the situation vastly improves. If you know you can go to jail if your postdocs fabricate data, I bet you will be a lot more thorough in checking all the original data. The problem of scientific research is misaligned incentives. There is tremendous upside for publishing novel ideas and very little downside for fraud, especially for the PI who can pawn it off on the overworked, underpaid post-doc.

dmckeon 3y ago
Rather than an post-hoc investigation model, how about promoting reproducibility in publication, with an expectation that others can run the same study with identical equipment, models, codes, playbooks, etc. For PIs getting government funding, true reproducibility should be a requirement, and other researchers should be funded to report pro or con on the reproducibility of studies, and then ethical lapses could be much more visible. Science based solely on faith in the ethics of other researchers is just religion with more complicated rituals.
- c0pium 3y ago
  Reproduction is not valued by the community in any way that matters. There’s no prize for reproducing, or failing to reproduce, others work. Getting a grant to do reproduction is much harder than getting the initial grant. Nobody is citing reproduction papers, so their authors can’t get that sweet sweet citation count. We as a society don’t actually value the discipline of science or the pursuit of knowledge. We value being first.
  - staunton 3y ago
    > We as a society don’t actually value the discipline of science or the pursuit of knowledge. The discipline of science is exactly how you get knowledge in a society that doesn't value its pursuit. It's an imperfect process of imperfect individuals that usually nevertheless arrives at knowledge eventually. Fraud is just another part of this process. Additional checks for fraud or errors cost time and money. Effective checks would basically mean science costs 2x as much. How much fraud can we tolerate? Is it worth spending this much to reduce the amount of fraud compared to investing in more research? Can we slightly change incentives (as opposed to inventing some utopia that cannot be implemented) and thereby reduce fraud more cheaply? The academic discipline has been changing ever since and will continue to change and progress.
    - dmckeon 3y ago
      Publishing reproducible research would disincentivize fraud by providing a clear way to expose fraud, and could tend to benefit other researchers in the field, by effectively sharing specific methods as well as results. Incentivizing reproducibility would be hard as a grassroots effort, but would be enforcible if it were tied to funding. Even if only a small percentage of published studies were reproduced, and only a small percentage of them were found to be fraudulent, the disincentive for fraud would be present. We do not need more Andrew Wakefields. > Effective checks would basically mean science costs 2x as much. By that logic, we would need to enforce speed limits by having one police cruiser following every driver.

dmckeon 3y ago

Rather than an post-hoc investigation model, how about promoting reproducibility in publication, with an expectation that others can run the same study with identical equipment, models, codes, playbooks, etc. For PIs getting government funding, true reproducibility should be a requirement, and other researchers should be funded to report pro or con on the reproducibility of studies, and then ethical lapses could be much more visible. Science based solely on faith in the ethics of other researchers is just religion with more complicated rituals.

c0pium 3y ago
Reproduction is not valued by the community in any way that matters. There’s no prize for reproducing, or failing to reproduce, others work. Getting a grant to do reproduction is much harder than getting the initial grant. Nobody is citing reproduction papers, so their authors can’t get that sweet sweet citation count. We as a society don’t actually value the discipline of science or the pursuit of knowledge. We value being first.
- staunton 3y ago
  > We as a society don’t actually value the discipline of science or the pursuit of knowledge. The discipline of science is exactly how you get knowledge in a society that doesn't value its pursuit. It's an imperfect process of imperfect individuals that usually nevertheless arrives at knowledge eventually. Fraud is just another part of this process. Additional checks for fraud or errors cost time and money. Effective checks would basically mean science costs 2x as much. How much fraud can we tolerate? Is it worth spending this much to reduce the amount of fraud compared to investing in more research? Can we slightly change incentives (as opposed to inventing some utopia that cannot be implemented) and thereby reduce fraud more cheaply? The academic discipline has been changing ever since and will continue to change and progress.
  - dmckeon 3y ago
    Publishing reproducible research would disincentivize fraud by providing a clear way to expose fraud, and could tend to benefit other researchers in the field, by effectively sharing specific methods as well as results. Incentivizing reproducibility would be hard as a grassroots effort, but would be enforcible if it were tied to funding. Even if only a small percentage of published studies were reproduced, and only a small percentage of them were found to be fraudulent, the disincentive for fraud would be present. We do not need more Andrew Wakefields. > Effective checks would basically mean science costs 2x as much. By that logic, we would need to enforce speed limits by having one police cruiser following every driver.