5 ms·
In corner A, a published and peer reviewed N=2408 study that tracks political tolerance and its stability over several years across two countries. In corner B,
by dTal 4y ago
In corner A, a published and peer reviewed N=2408 study that tracks political tolerance and its stability over several years across two countries.
In corner B, your personal qualitative impression of your immediate acquaintance group.
Is it possible that your skepticism is itself an example of overconfidence?
- throwoutway 4y agoSkepticism can lead to hypothesis which can lead to new studies which can be superior to previous studies
- dTal 4y agoSure. There's nothing wrong with saying "this is counterintuitive to me, and I think it's worth exploring with an open mind why my mental model and the study diverge." But it's not really an interesting contribution to the dialogue to say "meh, I don't buy it" without any kind of followup. What are we meant to do with that?
- Godel_unicode 4y agoGood job that’s not what they did then, I suppose. They gave examples of why they’re skeptical, theories about what might be causing those discrepancies, and proposed an experiment for how to tease that out. You are meant to engage with what they actually wrote.
- tomlockwood 4y agoN=2408 members of the Danish military or mechanical Turk participants, in a paper with one citation but a huge amount of retweets, in the time of the replication crisis. How is this paper credible?
- naasking 4y agoIn the replication crisis, psychology and sociology papers replicated about 30-35% of the time IIRC. In your experience, do random internet comments have a higher or lower chance of being true?
- tomlockwood 4y agoAs a random internet commenter, I think I face a bit of a paradox when answering this question. Do you, a random internet commenter, think this is a good paper? Should I believe you and the other people here? Should I look at how many upvotes this has on HN and see it as a sign of credibility, when all the upvotes come from random internet people?
- naasking 4y agoI think evaluating credibility by placing a paper in the evidence pyramid is generally a good idea. I don't think the replication crisis is a reason to outright dismiss a paper, but certainly skepticism before replication is always warranted. I think one should also always be skeptical of how universal one's own experience truly is.
- tomlockwood 4y agoI think I'd place this paper alone only a hair above someone's qualitative judgement, on the evidence pyramid, to be honest.
- naasking 4y agoI don't know. It features thousands of data points from different groups in two different countries showing the same trend. That's substantially more compelling than anecdotes IMO, even after tempering expectations due to the replication crisis, but to each their own.
- tomlockwood 4y agoThe only reason we're seeing it is because a largely untrained audience decided to upvote it, because it matches their anecdotal experience enough that they think it's a good paper.