3 ms·

> But merely been less confident (that they are) in their thoroughness and in their preclusion of contamination, is much easier to claim and be correct while do

> But merely been less confident (that they are) in their thoroughness and in their preclusion of contamination, is much easier to claim and be correct while doing so. But it's a trivial observation, and one that isn't actionable or testable. For the team to have designed and performed the instruments, experiments, data collection, and analysis to say "We're confident in our results and we have ruled out contamination" is substantive and is backed by data and actual experiments. For someone else to say, "I'm not so sure, I think it could have been contaminated," and then provide no actual argument or data on what could have gone wrong, is probably not a novel observation or comment. In addition, it's an easy thing to say (as you correctly point out) without putting oneself on the hook to show how or why. Here's what I mean: we can confidently say about every paper ever published that "I'm not so sure." Here are the possible outcomes: (a) the paper stands the test of time, the result holds, and nobody remembers anyone who said "I'm not so sure," or (b) someone finds a flaw in the paper, works with the authors and they publish a retraction or revised paper, and then whoever said "I'm not so sure," can easily (as you say) point out how "they called it," all without doing any actual work. I'd suggest reading every paper critically, realizing (as most people do) that the results are tentative, and either letting it go at that, or dig in and do the hard work to figure out what may be flawed in the work presented. Then work with the authors or publish a paper of one's own pointing out the flaw and the new resulting analysis.

coldtea 4y ago
>But it's a trivial observation, and one that isn't actionable or testable Yes. But if they make a bold claim that "contamination can be ruled out", it's worth it to say to folks ready to take them for their word that, "it's still a possibility", even if it's not a testable finding but a suggestion. Sure 'we can confidently say about every paper ever published that "I'm not so sure."'. But that doesn't mean we do say it for every paper ever published, or that we say it with equal weight. The alternative is taking every research announcement at its word, and acting like it's true until it's taken down by another paper. And even then, why'd we trust the second paper refuting the first got it right? In the end, it amounts to "if you didn't do the work or the review yourself, and din't find any specific actionable issue with it, then you should have no opinion about the probability of it being true or not". Which is quite problematic. In the end, we often need and should be able to make a summary assessment of the possibility of a paper's conclusions being right or not (as based on our layman experience), even without working with the data, going through it with a fine comb or trying to replicate it, but based on a sense of probability of the conclusion, our knowledge of past assurances in similar statements being ultimately proven wrong, potential conflicts of interest, and so on.
- craftsman 4y ago
  > then you should have no opinion about the probability of it being true or not Rather, I said when we read papers we should realize that results are tentative. It's fine to have some personal assessment of the probability, but it's not actionable by anyone, and communicating it doesn't assist in improving our collective assessment of what is going on in the world. I'd also suggest that we probably don't, in fact, "often need and should be able to make a summary assessment..."

coldtea 4y ago

>But it's a trivial observation, and one that isn't actionable or testable Yes. But if they make a bold claim that "contamination can be ruled out", it's worth it to say to folks ready to take them for their word that, "it's still a possibility", even if it's not a testable finding but a suggestion. Sure 'we can confidently say about every paper ever published that "I'm not so sure."'. But that doesn't mean we do say it for every paper ever published, or that we say it with equal weight. The alternative is taking every research announcement at its word, and acting like it's true until it's taken down by another paper. And even then, why'd we trust the second paper refuting the first got it right? In the end, it amounts to "if you didn't do the work or the review yourself, and din't find any specific actionable issue with it, then you should have no opinion about the probability of it being true or not". Which is quite problematic. In the end, we often need and should be able to make a summary assessment of the possibility of a paper's conclusions being right or not (as based on our layman experience), even without working with the data, going through it with a fine comb or trying to replicate it, but based on a sense of probability of the conclusion, our knowledge of past assurances in similar statements being ultimately proven wrong, potential conflicts of interest, and so on.

craftsman 4y ago
> then you should have no opinion about the probability of it being true or not Rather, I said when we read papers we should realize that results are tentative. It's fine to have some personal assessment of the probability, but it's not actionable by anyone, and communicating it doesn't assist in improving our collective assessment of what is going on in the world. I'd also suggest that we probably don't, in fact, "often need and should be able to make a summary assessment..."