3 ms·

The whole point for randomized trials, and much of modern statistics is to argue from ignorance. You hypothesize that maybe there's something missing, who knows

The whole point for randomized trials, and much of modern statistics is to argue from ignorance. You hypothesize that maybe there's something missing, who knows. A causal boogey man. Then you design a test robust to that. One term for problems this leads to is omitted variable bias. It's not about "ooh maybe this one thing is bad." It's about "this class of problem is introduced." It's tough to make an intuitive story for why RCTs are important, so the story sounds elaborate. You don't need to offer evidence against the theory to say that the methodology is weak. Anyways, wet streets might cause rain.

AstralStorm 7y ago
The problem being, RCTs are tricky to set up for social variables to have enough statistical power, while avoiding the "test bias" which is known to change behavior. (Essentially a kind of placebo effect.) It has to be huge. And how do you produce a social placebo to detect such effects by comparison? No treatment is not the same as known fake treatment. In fact, even the observational study here is quite underpowered and not multicultural. It is to be taken with a few bags of salt.
solidsnack9000 7y ago
Argument from ignorance is a specific kind of logical fallacy. It’s where you argue for something by saying, in effect, “we don’t know that it’s not true”. If a hypothesis leads you to test for something and collect (dis)confirming evidence, And your argument hinges on the evidence, then we aren’t arguing from ignorance.
- 6gvONxR4sf7o 7y ago
  It seems like you're saying "this is weak methodology" is a fallacy. It's open to all sorts of errors, but since we don't know that any of the errors are really the case, so we shouldn't argue that it's bad evidence? We aren't arguing boolean true/false, but strength of evidence.
  - solidsnack9000 7y ago
    There are many ways to argue for something like "this is weak methodology" and some of them are fallacies. Some of them aren't. A good argument against a study's methodology would probably talk a lot more about the methodology, and a lot less about a believable alternative narrative that is supported by the study. The latter strategy is accepting of the methodology because it is granting that the observed effect is real and deserving of explanation. In a different light, the earlier comment ("As an example of how the observed differences could still be due to nurture rather than nature...") is kind of weird because it is as though the effect might as well be due to nature -- it would present the same way and be as consistent and robust -- even if it were due to socialisation.

AstralStorm 7y ago

The problem being, RCTs are tricky to set up for social variables to have enough statistical power, while avoiding the "test bias" which is known to change behavior. (Essentially a kind of placebo effect.) It has to be huge. And how do you produce a social placebo to detect such effects by comparison? No treatment is not the same as known fake treatment. In fact, even the observational study here is quite underpowered and not multicultural. It is to be taken with a few bags of salt.

solidsnack9000 7y ago

Argument from ignorance is a specific kind of logical fallacy. It’s where you argue for something by saying, in effect, “we don’t know that it’s not true”. If a hypothesis leads you to test for something and collect (dis)confirming evidence, And your argument hinges on the evidence, then we aren’t arguing from ignorance.

6gvONxR4sf7o 7y ago
It seems like you're saying "this is weak methodology" is a fallacy. It's open to all sorts of errors, but since we don't know that any of the errors are really the case, so we shouldn't argue that it's bad evidence? We aren't arguing boolean true/false, but strength of evidence.
- solidsnack9000 7y ago
  There are many ways to argue for something like "this is weak methodology" and some of them are fallacies. Some of them aren't. A good argument against a study's methodology would probably talk a lot more about the methodology, and a lot less about a believable alternative narrative that is supported by the study. The latter strategy is accepting of the methodology because it is granting that the observed effect is real and deserving of explanation. In a different light, the earlier comment ("As an example of how the observed differences could still be due to nurture rather than nature...") is kind of weird because it is as though the effect might as well be due to nature -- it would present the same way and be as consistent and robust -- even if it were due to socialisation.