4 ms·

> The effect of opposition on the binarized full set of 34 objective knowledge items variable: > Estimate Std. Error df t value Pr(>|t|) > (Intercept) 25.9059

• somenameforme 4y ago
  Like mentioned, I was more interested in the impact on the subtopic specific results because of what we discussed. Giving a quiz on American history to judge your knowledge of French history is obviously not a reasonable idea, even if the skill-set overlap would probably give at least some weak correlation. In the field-specific binarized case (which removes the -3 to +3 noise), the impact is 0.09 points. So that translates to a fraction of a question difference in results. And while I'm aware trimming extreme outliers, regardless of the side they end up on, I'm unaware of any study entirely removing a segment from their sample, from one side only, which is critical to your entire hypothesis, and doing so without any explanation, let alone justification, whatsoever. One thing I'd observe is that the observed effect is small enough that if this culled group had a knowledge specific score below the mean, then it's likely that their entire conclusion would be invalid. One other issue we have not discussed is the methodology not only resulted in a very non-representative sample, but also was indirectly testing something else. The surveys were done on the internet, and all of the general knowledge questions (besides the covid ones, which were just...) have answers which can be looked up in a matter of seconds.
  • mlyle 4y ago
    > I was more interested in the impact on the subtopic specific results because of what we discussed. I think they're both interesting. Broad overconfidence in performance in basic science combined with low performance is an interesting characteristic of a population. The fact that this is represented in those with contrarian views is interesting. > So that translates to a fraction of a question difference in results. The subscale finding, just shows that this same phenomenon appears to also extend to the subjects they are contrarian about. And it's about the same magnitude of effect, because there's very few questions on the subscale. On the big test, a participant does about 2% worse per unit of disagreement. On the subscale, they do about 1.8% worse per unit of disagreement. The effect appears to have the same magnitude and is statistically significant in both cases. This is an invitation to do studies on specific fields of disagreement with better samples and more questions on the subscale. But this early research casts a wide net across many types of anti-consensus view. Study 5 is a small, possibly flawed step in this direction. I'd be particularly interested in a 2 variable analysis that attempts to model how one's overall objective performance and level of disagreement models objective performance in the specific field where they disagreed. > outliers, ... This is all moving goalposts when we were originally discussing the question set. I've already agreed that sociology and psychology research via Turk is problematic and faces many confounds and selection problems.
    • somenameforme 4y ago
      Something I recently considered was that many of the questions, the "good" questions, do ultimately test ideology more than knowledge. The obvious example would be something like "Humans share the majority of their genes with chimps." This is of course factually true true, but it seems much more likely to test ideology than scientific knowledge, in part because of how it is phrased. Imagine the question were framed as "Scientists claim that humans share the majority of their genes with chimps." Now you are no longer testing agreement with the scientific view, but knowledge of it. And perhaps a bigger issue is that there were literally zero questions that would do the same "trick" vice-versa for somebody who ideologically agrees with one of the topics, but otherwise lacked much knowledge. Testing this would have been quite easy by simply throwing in questions that sound like they fit the consensus but have an important aspect that makes them false. An example would be "Evolution is a process of improving a species which, over time, may lead one species to become an entirely new one." That is, of course, false - but somebody of low knowledge and high consensus agreement, would likely consider it to be true. My personal hypothesis would be that extreme views tend to be associated with high confidence and relatively low knowledge. And this could be extremes of agreement or disagreement with a topic. As Betrand Russell put it, "The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt."
      • mlyle 4y ago
        > This is of course factually true true If it's factually true, it's not testing ideology. > but it seems much more likely to test ideology than scientific knowledge, in part because of how it is phrased Is there anyone credible who disagrees with the statement here? I think a bigger issue is that it may be testing not knowledge but careful, critical reading. I can score well on this test, but my flippant quick answers are not very accurate. I am not surprised those that are worse at critical reading might be more likely to have anti-consensus views. > My personal hypothesis would be that extreme views tend to be associated with high confidence and relatively low knowledge. That doesn't seem to be the case here; while we don't know the "1" full agreement results, the rest seems to be a pretty dang linear trend line.