4 ms·
Argumentum ad populum my friend. It is absolutely worth making the questions more clear, because this is the entire point of the study. Do people who do not ag
by somenameforme 4y ago
Argumentum ad populum my friend.
It is absolutely worth making the questions more clear, because this is the entire point of the study. Do people who do not agree with the consensus on something lack knowledge, or do simply not agree with said knowledge? Conflating the two in your own study does nothing but set yourself up to observe a tautology (those who disagree with a consensus, disagree with a consensus), which a cynic may interpret as not entirely accidental.
And no, I'm not referencing statistical power. If I haven't made myself clear, I think this study and their numbers, are both going to fall well into the endless black hole that is the replication crisis, which is especially pronounced in social psychology - which has an aggregate replication success rate in the 20s. What I am saying is that you can't make your broad claim based even what I suspect are deeply massaged figures.
- mlyle 4y ago> Argumentum ad populum my friend. Sorry, no. Overwhelming agreement along with readily self-observable characteristics is good enough. You can claim that the kidney is actually the largest organ of the human body, but everyone else agreeing is good enough-- especially when we can look at pictures from other people we trust, arrange to check a cadaver ourselves if we really care, etc. I mean, I guess it's possible Big Skin (tm) has rigged all the measures and faked everything. /s > What I am saying is that you can't make your broad claim based even what I suspect are deeply massaged figures. One submeasure on one subpopulation having an outcome that does not support the claim but is also not inconsistent with that claim doesn't invalidate the claim. You are talking about: * A non-statistically significant finding * Showing an opposite slope, but of tiny magnitude compared to the slopes in the other direction * On one subpopulation * On one submeasure.
- somenameforme 4y agoWhen Galileo made his case for a heliocentric universe, there was no silver bullet he offered. He simply personally felt it to be more probable than the geocentric model in spite the overwhelming evidence to the contrary given the evidence of the time. Would you thus claim he was lacked knowledge (of the geocentric universe) if he chose to respond false on a question "True/False Everything in the universe revolves around the Earth?" Measuring agreement is different than measuring knowledge. As for your hypothesis, what would you propose would be a null hypothesis to test your hypothesis? It seems to me that it would be "Somebody who opposes a scientific consensus will score about the same in a test of general knowledge as somebody who supports it." And the climate change example confirms that null hypothesis, which would thus reject your hypothesis.
- mlyle 4y agoI'm getting less and less interested in galloping around talking about different minutiae. I basically disagree with your entire argument. But: > And the climate change example confirms that null hypothesis, which would thus reject your hypothesis. ???! fails to reject the null hypothesis. This seems to be a rather large and fundamental error. You don't ever "confirm[ the] null hypothesis"-- and you certainly don't in a way that lets you exclude other explanations. e.g. I flip a coin twice. It is heads once and tails once. I perform a statistical test and this outcome is consistent with random chance. In no way does this let me reject the idea that the coin is unfair. (Indeed, doing a trial of 4 flips and getting 4 heads in a row is consistent with random chance; such a trial would not be likely to increase my belief that the coin is fair, even if it is not strong statistical evidence that the coin is unfair). Back to our case: we have a whole lot of strong results in one direction, and one ambiguous near-zero result in one submeasure in one subpopulation. Perhaps that subpopulation really is different. It's an interesting thing to measure again. But if you measure enough things, you can expect to find some of these in your result set purely by chance.
- somenameforme 4y agoWith all due respect, that feels like a dodge. I decided to look at the result data for this survey [1]. I was unsurprised to find, in a simple skim of only the first survey, at least 2 comments mentioning this issue we're discussing. Asking an e.g. creationist "True/False The Earth is about 4.5 billion years old." and "True/False Scientists claim that the Earth is about 4.5 billion years old." are questions that are going to get very different answers. One measures assent with consensus, one measures knowledge of it. At least 2 people actually manually typing the issue out in a relatively small sample (for this sort of arbitrary specificity) means this is an extremely significant issue. My argument, in the most specific form, is that I do not believe this study measures, or was intended to measure, what the authors' claim it does. This issue is quite subtle, but I'm inclined to believe in malice over incompetence when these authors have spent years working these numbers [2]. The current study is a rehash of a study they did two years ago. The max assent group does indeed demonstrate reduced knowledge / increased self perceived knowledge. And there are demographic biases abounds. The desired trend varies by an order of magnitude in countries outside the US. So this study took the most extreme sample from a subpopulation they were already familiar with the results of, stripped out the subpopulation of that subpopulation which challenged their hypothesis, added in a number of questions that were more likely to result in noise than anything else (the population of those taking surveys for pennies a piece on Amazon Turk, that can explain the difference between e.g. UVA and UVB radiation is certainly near zero so true/false just gives you noise and randomness), and then published. Another fun sample bias: Literally every single person who took fewer than 3 minutes to complete the survey (again based on survey 1) and bothered to fill in some (generally random) answers indicated max dissent. An interesting issue in itself, but a further problem in this study given relatively few people indicated max dissent, so they're measuring people literally racing through the survey for their $0.85 likely without reading it beyond looking for the "Please select #2 as the answer to this question." questions. [1] - https://osf.io/x23c8/?view_only=29a92a9a707547149f210e5bf76a6b54 https://osf.io/x23c8/?view_only=29a92a9a707547149f210e5bf76a... [2] - https://osf.io/t82j3/ https://osf.io/t82j3/