5 ms·

Only if you believe that work should not be evaluated on its own merits. If the methodology is good, the sample sizes big enough, the statistics correctly done

Only if you believe that work should not be evaluated on its own merits. If the methodology is good, the sample sizes big enough, the statistics correctly done, etc., I don't care if Mickey Mouse wrote the study. The climate change example is bad because oil companies have been actively suppressing correct evidence showing the existence of climate change since the 1980s. I'm less familiar with the tobacco studies, but I wouldn't be surprised if they've been doing the same thing, just on longer time scales.

vlovich123 4y ago
Most people tend to try to engage the world with how it is rather than how they’d like it to be. It’s definitely not as simple as you make it out to be. They employ both strategies - they try to suppress real studies and they try to generate bullshit on top to confuse them situation / overwhelm the capacity of the system to deal with it. Sure it’s an ad hominem attack to use past behavior to judge someone’s current or actions statements and taking actions as a result (eg ignoring them). And yet, is that a logical fallacy? I don’t think so but you seem to have an opposite conclusion.
- Paul-Craft 4y ago
  I'm tempted to say that claiming it's naivete to expect people to do work to evaluate a research paper is naive, as well as ad hominem. If you don't have the time or expertise to do so, then do the rest of us a favor and don't try. Now, if you're talking about using such criteria as a heuristic to decide which papers to put effort into evaluating, then we have another discussion to have.
  - vlovich123 4y ago
    That’s exactly what I’m saying. I don’t bother trying to evaluate a paper because a) my area of expertise only extends so far b) papers are largely worthless in their ability to communicate where the research done was valid. The replication crises and many many fraudulent papers surely would convince anyone that there’s a real problem with our current system. Maybe you’re a brilliant mind that can separate the wheat from the chaff. Most people, even leading researchers in the field, have trouble though so I don’t feel like I’m in bad company.
Anthony-G 4y ago
> If the methodology is good, the sample sizes big enough, the statistics correctly done, etc., I don't care if Mickey Mouse wrote the study. I’d agree with this but in the real world, we usually don’t and can’t really know if the methodology and statistical analysis was solid and rigorously followed as described in the paper – if it they were even adequately described. The only way to be sure is to replicate the study and that costs time and money (as per Brandolini's Law). Evaluating incentives (e.g., follow the money) is a useful heuristic to evaluate the creditability and trustworthiness of a particular study.
alasarmas 4y ago
I am sympathetic with your point of view, but I find myself in a position of what Scott Alexander calls “epistemic learned helplessness”[0]. My powers of understanding the correct method of scientific research are limited. I understand that there must be some studies that are done correctly, but I don’t know which ones. I assume that these are the older studies that haven’t been refuted or retracted after a long time. So I end up trying to trust new research that doesn’t seem to stem from a conflict of interest… although it makes sense to me that things that researchers have some interest in, somehow, are the things that they study. It’s a very neat trap I find myself in! Nothing for it but to become intimately familiar with the methods of scientific research? 0. https://slatestarcodex.com/2019/06/03/repost-epistemic-learned-helplessness/ https://slatestarcodex.com/2019/06/03/repost-epistemic-learn...

vlovich123 4y ago

Most people tend to try to engage the world with how it is rather than how they’d like it to be. It’s definitely not as simple as you make it out to be. They employ both strategies - they try to suppress real studies and they try to generate bullshit on top to confuse them situation / overwhelm the capacity of the system to deal with it. Sure it’s an ad hominem attack to use past behavior to judge someone’s current or actions statements and taking actions as a result (eg ignoring them). And yet, is that a logical fallacy? I don’t think so but you seem to have an opposite conclusion.

Paul-Craft 4y ago
I'm tempted to say that claiming it's naivete to expect people to do work to evaluate a research paper is naive, as well as ad hominem. If you don't have the time or expertise to do so, then do the rest of us a favor and don't try. Now, if you're talking about using such criteria as a heuristic to decide which papers to put effort into evaluating, then we have another discussion to have.
- vlovich123 4y ago
  That’s exactly what I’m saying. I don’t bother trying to evaluate a paper because a) my area of expertise only extends so far b) papers are largely worthless in their ability to communicate where the research done was valid. The replication crises and many many fraudulent papers surely would convince anyone that there’s a real problem with our current system. Maybe you’re a brilliant mind that can separate the wheat from the chaff. Most people, even leading researchers in the field, have trouble though so I don’t feel like I’m in bad company.

Anthony-G 4y ago

> If the methodology is good, the sample sizes big enough, the statistics correctly done, etc., I don't care if Mickey Mouse wrote the study. I’d agree with this but in the real world, we usually don’t and can’t really know if the methodology and statistical analysis was solid and rigorously followed as described in the paper – if it they were even adequately described. The only way to be sure is to replicate the study and that costs time and money (as per Brandolini's Law). Evaluating incentives (e.g., follow the money) is a useful heuristic to evaluate the creditability and trustworthiness of a particular study.

alasarmas 4y ago

I am sympathetic with your point of view, but I find myself in a position of what Scott Alexander calls “epistemic learned helplessness”[0]. My powers of understanding the correct method of scientific research are limited. I understand that there must be some studies that are done correctly, but I don’t know which ones. I assume that these are the older studies that haven’t been refuted or retracted after a long time. So I end up trying to trust new research that doesn’t seem to stem from a conflict of interest… although it makes sense to me that things that researchers have some interest in, somehow, are the things that they study. It’s a very neat trap I find myself in! Nothing for it but to become intimately familiar with the methods of scientific research? 0. https://slatestarcodex.com/2019/06/03/repost-epistemic-learned-helplessness/ https://slatestarcodex.com/2019/06/03/repost-epistemic-learn...