4 ms·
Via HN yesterday [1]- an editor of _Anaesthesia_ did a meta study of the papers he handled that conducted RCTs. He had data from 150 of them and concluded: >
by kansface 3y ago
Via HN yesterday [1]- an editor of _Anaesthesia_ did a meta study of the papers he handled that conducted RCTs. He had data from 150 of them and concluded:
> ...26% of the papers had problems that were so widespread that the trial was impossible to trust, he judged — either because the authors were incompetent, or because they had faked the data.
This is not a one off.
[1] https://www.nature.com/articles/d41586-023-02299-w https://www.nature.com/articles/d41586-023-02299-w
- tptacek 3y agoI didn't say it was a one-off. But 150 papers is, to a first approximation, a one-off of all the science done in a given year. We produce millions of journal articles every year.
- Auracle 3y agoMaybe we shouldn't. What's the point of all of that data if a good portion of it can't be trusted?
- tptacek 3y agoHere we're talking about a proportion significantly less than 1%.
- esyir 3y agoThere's something to be said about a defense of this that doesn't account for random sampling. Assuming that they did a proper sample of said papers, that implies that for whatever domain they sampled, 26% is likely a decent estimate of actual issues. Increasing the scale doesn't make a proportional estimate any better.