3 ms·

The authors have done the work for me? Hmmm. I don't think that's how it works. In order for me to evaluate any particular paper, I have to read it myself and e

by jongalt1962 3y ago

The authors have done the work for me? Hmmm. I don't think that's how it works. In order for me to evaluate any particular paper, I have to read it myself and evaluate it myself. I have to identify implicit premises, I have to ask, did they follow the scientific method? Does their data make sense? Are they missing alternative explanations? Does their conclusion match their initial hypothesis and/or their data? And many other questions. No one can do that for me. And as I said in another comment, there are too many corrupt incentives to produce predetermined outcomes. Thanks for providing the links, I expect that they will be enlightening, in one way or another. Please let me know how you determine the validity of scientific paper.

dekhn 3y ago
The best way to determine the validity of a paper is to give it to a team of grad students in a journal club. They will immediately find large numbers of flaws (in any paper), some of which may invalidate the conclusion. Papers which survive multiple rounds of attempts to invalidate the conclusion, are less likely to be incorrect. You can't just naively go in and ask "did they follow the scientific method" or "does the data make sense", etc, unless you understand the field in enough detail to not repeat the copious mistakes of past generations. For example, this study is retrospective, while the RECOVERY study was designed to be prospective (if I read their study design correctly; I'm not an epidemiologist, my phd is in biophysics). In general, retrospective studies produce less convincing evidence and so if yuou asked me to compare the results of the two studies, I would place more weight on RECOVERY. IIUC it also has a larger sample size. But all these things pale in comparison to complex, detailed analysis of the study plans to identify any major mistakes.
cantSpellSober 3y ago
> I have to read it myself Hard work, right? How did you determine the validity of the "couple of [unnamed] doctors" that you trust over published research? > "science" these days is corrupted by [...] I am suspicious of all of it Including the science done by the "couple of doctors" you mentioned? You believe "they could've stopped the pandemic early on" and you're suspicious of them?

dekhn 3y ago

The best way to determine the validity of a paper is to give it to a team of grad students in a journal club. They will immediately find large numbers of flaws (in any paper), some of which may invalidate the conclusion. Papers which survive multiple rounds of attempts to invalidate the conclusion, are less likely to be incorrect. You can't just naively go in and ask "did they follow the scientific method" or "does the data make sense", etc, unless you understand the field in enough detail to not repeat the copious mistakes of past generations. For example, this study is retrospective, while the RECOVERY study was designed to be prospective (if I read their study design correctly; I'm not an epidemiologist, my phd is in biophysics). In general, retrospective studies produce less convincing evidence and so if yuou asked me to compare the results of the two studies, I would place more weight on RECOVERY. IIUC it also has a larger sample size. But all these things pale in comparison to complex, detailed analysis of the study plans to identify any major mistakes.

cantSpellSober 3y ago

> I have to read it myself Hard work, right? How did you determine the validity of the "couple of [unnamed] doctors" that you trust over published research? > "science" these days is corrupted by [...] I am suspicious of all of it Including the science done by the "couple of doctors" you mentioned? You believe "they could've stopped the pandemic early on" and you're suspicious of them?