3 ms·
I think if the claims that article makes are both (i) true and (ii) meaningful (as opposed to an annoying artifact of the norms of how these fields operate) we'
by tptacek 2y ago
I think if the claims that article makes are both (i) true and (ii) meaningful (as opposed to an annoying artifact of the norms of how these fields operate) we'd be seeing downstream effects of it that we do not see†. I think this is a particularly weird time in the history of biomedical science to be trying to further the argument that everybody is adding cards to a giant house of cards, but you do you.
Either way: people are responding to specific stories about fraudulent research, and my point is that the numerator of those stories is infinitesimal compared to the denominator of how many papers are published in these fields annually, by how many separately-funded R1 research teams.
I'm really not interested in being boxed into a debate about whether p-hacking matters, or scientific fraud exists. Fraud is bad! I'll just keep reiterating that the optimal level of fraud in a large system is almost never zero.
† That article is about a bunch of different fields, and I'm commenting only on a very specific subset of them; fuck do I know how well social science is operating right now
- ziddoap 2y ago>I'll just keep reiterating that the optimal level of fraud in a large system is almost never zero. I've been trying to think about what you meant since you wrote this earlier in another comment, and I'm having some trouble wrapping my head around it so I think I'll just ask (if you'd be so kind to explain). What makes some level of fraud more optimal than zero fraud?
- tptacek 2y agoPatrick McKenzie popularized that wording, but I think he may have taken inspiration from Dan Davies, and the idea is older still. It's simply that fraud is endemic to large systems, and past a certain threshold eliminating it makes people worse-off on balance as the countermeasures retard the efficient operation of the system. You can get to zero fraud in lending easily: just don't make loans. Grinding all scientific research to a halt to "true up" the statistical reporting used for otherwise valid and important research results would be a very good example of an intervention that would (a) "reduce fraud" and (b) leave us all significantly worse off.
- defrost 2y agoI'm in agreement with tptacek and P.McKenzie and feel it's something more generally true; perfection is the enemy of throughput. Whether it's mine processing, agriculture, bank lending, { insert domain } there are always a number of drags on profit ( fraud, equipment wear and tear, petty theft, holes in fences ) and generally always a threshold past which the cost of pursuit of perfection exceeds the decreasing marginal benefit.
- ziddoap 2y agoThat makes a lot more sense than the literal interpretation. Thanks.
- s1artibartfast 2y agoI think that is the literal interpretation. That is to say, Zero is the optimal in an imaginary system, where cows are spherical and you can make all the assumptions you want. Once you start working with real world systems with more and complex constraints, the simple story goes away, and perfection is not possible. The phrasing challenges the reader to question the model with which they are using to define optimal.
- ziddoap 2y agoWe have different definitions of literal, I guess. The first two definitions of "literal" are: "Conforming or limited to the simplest, nonfigurative, or most obvious meaning of a word or words. " "Word for word; verbatim." Taking the sentence in question verbatim would be "some fraud is optimal". The nuanced (non-literal) interpretation is what Thomas wrote.
- s1artibartfast 2y agoyes, some level of fraud is optimal in a system. It is literally true.
- 2y ago