3 ms·

> With all due respect, that feels like a dodge. I note you've completely moved on from the huge fallacy of a statistical claim that you made, to choose to cir

by mlyle 4y ago

> With all due respect, that feels like a dodge. I note you've completely moved on from the huge fallacy of a statistical claim that you made, to choose to circle back to hit one of the other topics we've already discussed earlier. We all know about the Gish gallop. Pick one. You basically committed data interpretation murder in the grandparent comment. If I fail to engage with every single point of yours, it doesn't make those points right. Especially when they're repeated after having already been addresses! > means this is an extremely significant issue. OK, so there's some questions on the whole test which might inadvertently measure religiosity-- if people don't realize they're supposed to respond with the scientific belief. (Of course, once they're betting on how they will score on that test, it's hard to understand what else they might believe would be the rubric for said test). Even so, this would hardly explains things showing up in the subscores with the same slopes, unless you think the bulk of the test is contaminated. You're the one pulling the CSVs and running analyses-- is that question missed more by people who have anti-consensus views? > An interesting issue in itself, but a further problem in this study given relatively few people indicated max dissent, so they're measuring people literally racing through the survey for their $0.85 likely without reading it beyond looking for the "Please select #2 as the answer to this question." questions. Look, it's just easy to eyeball the graphs and see removing the '7's does nothing to the slope of the lines.

somenameforme 4y ago
The "betting" was for $0.50, which was not lost if they failed. The only cost was not accepting an alternative $0.25 immediate bonus. There's this major issue of sampling only people doing online surveys for pennies a piece, and it is possible that this specific sample might meaningfully change their behavior for $0.50 (or $0.25, depending on how you look at it), but I wouldn't really assume as much. If they really wanted that extra quarter or two, they could have simply looked everything up online. Looking briefly over the data, anti-consensus views do seem to correlate well with questions that were basically just proxies for measuring that dissent again. For instance one question was 'man was alive at the same time as dinosaurs' type thing. The correct answer on 1-6 is a 1. People who rejected the evolution consensus scored 4.6 on it, with a mode of 7. The mean of all respondents was 3. The score of people who indicated max opposition to e.g. nuclear power was 2.85. Those samples were my first and not cherry picked, beyond intentionally picking a group that would likely have very different biases (opposition to evolution vs opposition to nuclear) to compare against. Imagine you simply framed the question slightly differently, but testing the exact same knowledge, "Most scientists do not believe humans were alive at the same time as the dinosaurs." I suspect you would have gotten radically different responses. And this issue (in various incarnations) is something I think that is absolutely destroying the 'soft sciences.' Quite small changes in experiment or survey can yield radically different results, and that can be exploited to "prove", more or less, whatever you want to. p.s. The reason I moved on is because it was simply poor phrasing in a casual anon chat.
- mlyle 4y ago
  > p.s. The reason I moved on is because it was simply poor phrasing in a casual anon chat. It wasn't poor phrasing. Being unable to reject the null hypothesis (on one subtest, even) doesn't mean there's no effect. If you test something 15 different ways, and you get a statistically significant effect with consistent slope on 12 of them, a non-statistically significant effect with a slope in the same direction on 2, and a non-statistically significant effect with a tiny slope in the opposite direction on 1... that looks pretty universal. It's an interesting question for followup research whether those 3 things are actually different in some way, or whether it was luck.
  - somenameforme 4y ago
    You look at a billion and one swans. For the first billion you see nothing but white swans, yet for the final one you look at you see a black swan. You now cannot say swans are universally white. All it takes is a single exception to break a rule. This is one of the many issues I have with social science - lack of falsifiability. Social science effectively lacks any degree of falsifiability simply because everything is based on vague probabilistic distributions which vary dramatically based on subtle nuances like asking an identical question two different ways, or testing on different populations in cases where the hypothesis is suggested to generalize - as here. If there was any notion of falsification, it would all be false. In many ways it's quite analogous to astrology which, if you are unaware, was indeed considered a science, studied in universities for hundreds of years. It's end, somewhat ironically, came not from scientific objection to the fact it suffered the same issues as the social sciences today, but from the Church who deemed it heretical in claims of defacto divination.
    - mlyle 4y ago
      > You look at a billion and one swans. For the first billion you see nothing but white swans, yet for the final one you look at you see a black swan. This analogy doesn't fit our circumstance. You take two photos each of 7 swans, a close-up and a wide-angle one. For all of the 7 wide-angle photos, the swan is obviously white. For the 7 close up-photos: you get 4 good close-up photos of white swan. Then there's 3 bad photos. 2 of them look probably white, and 1 looks like the swan is maybe dark grey. All the evidence you have is consistent with all the swans being white. Looking to see if you can find those 3 swans and take a better close look at them might be a nice bit of followup.

by mlyle 4y ago

somenameforme 4y ago

The "betting" was for $0.50, which was not lost if they failed. The only cost was not accepting an alternative $0.25 immediate bonus. There's this major issue of sampling only people doing online surveys for pennies a piece, and it is possible that this specific sample might meaningfully change their behavior for $0.50 (or $0.25, depending on how you look at it), but I wouldn't really assume as much. If they really wanted that extra quarter or two, they could have simply looked everything up online. Looking briefly over the data, anti-consensus views do seem to correlate well with questions that were basically just proxies for measuring that dissent again. For instance one question was 'man was alive at the same time as dinosaurs' type thing. The correct answer on 1-6 is a 1. People who rejected the evolution consensus scored 4.6 on it, with a mode of 7. The mean of all respondents was 3. The score of people who indicated max opposition to e.g. nuclear power was 2.85. Those samples were my first and not cherry picked, beyond intentionally picking a group that would likely have very different biases (opposition to evolution vs opposition to nuclear) to compare against. Imagine you simply framed the question slightly differently, but testing the exact same knowledge, "Most scientists do not believe humans were alive at the same time as the dinosaurs." I suspect you would have gotten radically different responses. And this issue (in various incarnations) is something I think that is absolutely destroying the 'soft sciences.' Quite small changes in experiment or survey can yield radically different results, and that can be exploited to "prove", more or less, whatever you want to. p.s. The reason I moved on is because it was simply poor phrasing in a casual anon chat.

mlyle 4y ago
> p.s. The reason I moved on is because it was simply poor phrasing in a casual anon chat. It wasn't poor phrasing. Being unable to reject the null hypothesis (on one subtest, even) doesn't mean there's no effect. If you test something 15 different ways, and you get a statistically significant effect with consistent slope on 12 of them, a non-statistically significant effect with a slope in the same direction on 2, and a non-statistically significant effect with a tiny slope in the opposite direction on 1... that looks pretty universal. It's an interesting question for followup research whether those 3 things are actually different in some way, or whether it was luck.
- somenameforme 4y ago
  You look at a billion and one swans. For the first billion you see nothing but white swans, yet for the final one you look at you see a black swan. You now cannot say swans are universally white. All it takes is a single exception to break a rule. This is one of the many issues I have with social science - lack of falsifiability. Social science effectively lacks any degree of falsifiability simply because everything is based on vague probabilistic distributions which vary dramatically based on subtle nuances like asking an identical question two different ways, or testing on different populations in cases where the hypothesis is suggested to generalize - as here. If there was any notion of falsification, it would all be false. In many ways it's quite analogous to astrology which, if you are unaware, was indeed considered a science, studied in universities for hundreds of years. It's end, somewhat ironically, came not from scientific objection to the fact it suffered the same issues as the social sciences today, but from the Church who deemed it heretical in claims of defacto divination.
  - mlyle 4y ago
    > You look at a billion and one swans. For the first billion you see nothing but white swans, yet for the final one you look at you see a black swan. This analogy doesn't fit our circumstance. You take two photos each of 7 swans, a close-up and a wide-angle one. For all of the 7 wide-angle photos, the swan is obviously white. For the 7 close up-photos: you get 4 good close-up photos of white swan. Then there's 3 bad photos. 2 of them look probably white, and 1 looks like the swan is maybe dark grey. All the evidence you have is consistent with all the swans being white. Looking to see if you can find those 3 swans and take a better close look at them might be a nice bit of followup.