5 ms·

> This is of course factually true true If it's factually true, it's not testing ideology. > but it seems much more likely to test ideology than scientific kn

by mlyle 4y ago

> This is of course factually true true If it's factually true, it's not testing ideology. > but it seems much more likely to test ideology than scientific knowledge, in part because of how it is phrased Is there anyone credible who disagrees with the statement here? I think a bigger issue is that it may be testing not knowledge but careful, critical reading. I can score well on this test, but my flippant quick answers are not very accurate. I am not surprised those that are worse at critical reading might be more likely to have anti-consensus views. > My personal hypothesis would be that extreme views tend to be associated with high confidence and relatively low knowledge. That doesn't seem to be the case here; while we don't know the "1" full agreement results, the rest seems to be a pretty dang linear trend line.

somenameforme 4y ago
Nobody is forced to accept a fact if they do not want to. There's a large gulf between awareness of knowledge, including facts, and agreement with such. This is an especially important nuance in this quiz, where the precise implications/meanings of knowledge vs ideology are a critical part of the study. Imagine that there was a secular Islamic expert. And he was quizzed on Islamic ideology in a similar fashion, such that he was obligated to imply belief in such, in his answers. Assuming he is being as accurate as he can in his responses, he would end up scoring as completely ignorant of Islam as he marked all beliefs as false. I disagree there's a clear trend here, even without the 1s, because there were clear exceptions such as e.g. climate change where, even by their metrics, anti-consensus views were predictive of greater knowledge. So you end up having to, at a minimum, limit the stated claim to specific fields.
- mlyle 4y ago
  > Nobody is forced to accept a fact if they do not want to. There's a large gulf between awareness of knowledge, including facts, and agreement with such. This is an especially important nuance in this quiz, where the precise implications/meanings of knowledge vs ideology are a critical part of the study. OK, well, here they expressed disagreement with stuff that is not just the scientific consensus, but the overwhelming agreement. If you want to phrase it as disagreeing rather than knowledge being lacking-- I think this is unhelpful. I do not think the nuance you're driving at here is worth complicating the questions or instructions. > because there were clear exceptions such as e.g. climate change No. All 7 areas of controversy did worse on the entire test. It was not statistically significant for climate change and evolution. It was highly statistically significant for 4 of the remaining 5. 6 of the 7 did worse on their on subscales (significantly significant for 4 of these 6), with effect sizes of -.83, -.65, -.28, -.82, -.88, -.55; climate change had a non-statistically significant finding of an effect with slope .03. I do not consider this a counterexample. You've been really advocating for study power and significance of results... when it favors your argument.
  - somenameforme 4y ago
    Argumentum ad populum my friend. It is absolutely worth making the questions more clear, because this is the entire point of the study. Do people who do not agree with the consensus on something lack knowledge, or do simply not agree with said knowledge? Conflating the two in your own study does nothing but set yourself up to observe a tautology (those who disagree with a consensus, disagree with a consensus), which a cynic may interpret as not entirely accidental. And no, I'm not referencing statistical power. If I haven't made myself clear, I think this study and their numbers, are both going to fall well into the endless black hole that is the replication crisis, which is especially pronounced in social psychology - which has an aggregate replication success rate in the 20s. What I am saying is that you can't make your broad claim based even what I suspect are deeply massaged figures.
    - mlyle 4y ago
      > Argumentum ad populum my friend. Sorry, no. Overwhelming agreement along with readily self-observable characteristics is good enough. You can claim that the kidney is actually the largest organ of the human body, but everyone else agreeing is good enough-- especially when we can look at pictures from other people we trust, arrange to check a cadaver ourselves if we really care, etc. I mean, I guess it's possible Big Skin (tm) has rigged all the measures and faked everything. /s > What I am saying is that you can't make your broad claim based even what I suspect are deeply massaged figures. One submeasure on one subpopulation having an outcome that does not support the claim but is also not inconsistent with that claim doesn't invalidate the claim. You are talking about: * A non-statistically significant finding * Showing an opposite slope, but of tiny magnitude compared to the slopes in the other direction * On one subpopulation * On one submeasure.
      - somenameforme 4y ago
        When Galileo made his case for a heliocentric universe, there was no silver bullet he offered. He simply personally felt it to be more probable than the geocentric model in spite the overwhelming evidence to the contrary given the evidence of the time. Would you thus claim he was lacked knowledge (of the geocentric universe) if he chose to respond false on a question "True/False Everything in the universe revolves around the Earth?" Measuring agreement is different than measuring knowledge. As for your hypothesis, what would you propose would be a null hypothesis to test your hypothesis? It seems to me that it would be "Somebody who opposes a scientific consensus will score about the same in a test of general knowledge as somebody who supports it." And the climate change example confirms that null hypothesis, which would thus reject your hypothesis.

by mlyle 4y ago

somenameforme 4y ago

Nobody is forced to accept a fact if they do not want to. There's a large gulf between awareness of knowledge, including facts, and agreement with such. This is an especially important nuance in this quiz, where the precise implications/meanings of knowledge vs ideology are a critical part of the study. Imagine that there was a secular Islamic expert. And he was quizzed on Islamic ideology in a similar fashion, such that he was obligated to imply belief in such, in his answers. Assuming he is being as accurate as he can in his responses, he would end up scoring as completely ignorant of Islam as he marked all beliefs as false. I disagree there's a clear trend here, even without the 1s, because there were clear exceptions such as e.g. climate change where, even by their metrics, anti-consensus views were predictive of greater knowledge. So you end up having to, at a minimum, limit the stated claim to specific fields.

mlyle 4y ago
> Nobody is forced to accept a fact if they do not want to. There's a large gulf between awareness of knowledge, including facts, and agreement with such. This is an especially important nuance in this quiz, where the precise implications/meanings of knowledge vs ideology are a critical part of the study. OK, well, here they expressed disagreement with stuff that is not just the scientific consensus, but the overwhelming agreement. If you want to phrase it as disagreeing rather than knowledge being lacking-- I think this is unhelpful. I do not think the nuance you're driving at here is worth complicating the questions or instructions. > because there were clear exceptions such as e.g. climate change No. All 7 areas of controversy did worse on the entire test. It was not statistically significant for climate change and evolution. It was highly statistically significant for 4 of the remaining 5. 6 of the 7 did worse on their on subscales (significantly significant for 4 of these 6), with effect sizes of -.83, -.65, -.28, -.82, -.88, -.55; climate change had a non-statistically significant finding of an effect with slope .03. I do not consider this a counterexample. You've been really advocating for study power and significance of results... when it favors your argument.
- somenameforme 4y ago
  Argumentum ad populum my friend. It is absolutely worth making the questions more clear, because this is the entire point of the study. Do people who do not agree with the consensus on something lack knowledge, or do simply not agree with said knowledge? Conflating the two in your own study does nothing but set yourself up to observe a tautology (those who disagree with a consensus, disagree with a consensus), which a cynic may interpret as not entirely accidental. And no, I'm not referencing statistical power. If I haven't made myself clear, I think this study and their numbers, are both going to fall well into the endless black hole that is the replication crisis, which is especially pronounced in social psychology - which has an aggregate replication success rate in the 20s. What I am saying is that you can't make your broad claim based even what I suspect are deeply massaged figures.
  - mlyle 4y ago
    > Argumentum ad populum my friend. Sorry, no. Overwhelming agreement along with readily self-observable characteristics is good enough. You can claim that the kidney is actually the largest organ of the human body, but everyone else agreeing is good enough-- especially when we can look at pictures from other people we trust, arrange to check a cadaver ourselves if we really care, etc. I mean, I guess it's possible Big Skin (tm) has rigged all the measures and faked everything. /s > What I am saying is that you can't make your broad claim based even what I suspect are deeply massaged figures. One submeasure on one subpopulation having an outcome that does not support the claim but is also not inconsistent with that claim doesn't invalidate the claim. You are talking about: * A non-statistically significant finding * Showing an opposite slope, but of tiny magnitude compared to the slopes in the other direction * On one subpopulation * On one submeasure.
    - somenameforme 4y ago
      When Galileo made his case for a heliocentric universe, there was no silver bullet he offered. He simply personally felt it to be more probable than the geocentric model in spite the overwhelming evidence to the contrary given the evidence of the time. Would you thus claim he was lacked knowledge (of the geocentric universe) if he chose to respond false on a question "True/False Everything in the universe revolves around the Earth?" Measuring agreement is different than measuring knowledge. As for your hypothesis, what would you propose would be a null hypothesis to test your hypothesis? It seems to me that it would be "Somebody who opposes a scientific consensus will score about the same in a test of general knowledge as somebody who supports it." And the climate change example confirms that null hypothesis, which would thus reject your hypothesis.