3 ms·

N=2408 members of the Danish military or mechanical Turk participants, in a paper with one citation but a huge amount of retweets, in the time of the replicatio

N=2408 members of the Danish military or mechanical Turk participants, in a paper with one citation but a huge amount of retweets, in the time of the replication crisis. How is this paper credible?

naasking 4y ago
In the replication crisis, psychology and sociology papers replicated about 30-35% of the time IIRC. In your experience, do random internet comments have a higher or lower chance of being true?
- tomlockwood 4y ago
  As a random internet commenter, I think I face a bit of a paradox when answering this question. Do you, a random internet commenter, think this is a good paper? Should I believe you and the other people here? Should I look at how many upvotes this has on HN and see it as a sign of credibility, when all the upvotes come from random internet people?
  - naasking 4y ago
    I think evaluating credibility by placing a paper in the evidence pyramid is generally a good idea. I don't think the replication crisis is a reason to outright dismiss a paper, but certainly skepticism before replication is always warranted. I think one should also always be skeptical of how universal one's own experience truly is.
    - tomlockwood 4y ago
      I think I'd place this paper alone only a hair above someone's qualitative judgement, on the evidence pyramid, to be honest.
      - naasking 4y ago
        I don't know. It features thousands of data points from different groups in two different countries showing the same trend. That's substantially more compelling than anecdotes IMO, even after tempering expectations due to the replication crisis, but to each their own.
        tomlockwood 4y ago
        The only reason we're seeing it is because a largely untrained audience decided to upvote it, because it matches their anecdotal experience enough that they think it's a good paper.
- dTal 4y ago
  As another random internet commenter, I anecdotally find that the least tolerant people I know are indeed substantially dumb. Thus, whatever probability you assign, between this comment and the original equally valid one contradicting it, we can derive no information, and must therefore rely on the study as the tiebreaker, however suspect we find it.
  - Godel_unicode 4y ago
    Some of the (outwardly?) most tolerant people I know are quite dumb, and I know many extremely intelligent people capable of tremendous feats of abstract thought who are both very tolerant and very intolerant. This is an unreplicated social science paper. It’s worth approximately the paper it isn’t printed on.

N=2408 members of the Danish military or mechanical Turk participants, in a paper with one citation but a huge amount of retweets, in the time of the replication crisis. How is this paper credible?

naasking 4y ago

In the replication crisis, psychology and sociology papers replicated about 30-35% of the time IIRC. In your experience, do random internet comments have a higher or lower chance of being true?

tomlockwood 4y ago
As a random internet commenter, I think I face a bit of a paradox when answering this question. Do you, a random internet commenter, think this is a good paper? Should I believe you and the other people here? Should I look at how many upvotes this has on HN and see it as a sign of credibility, when all the upvotes come from random internet people?
- naasking 4y ago
  I think evaluating credibility by placing a paper in the evidence pyramid is generally a good idea. I don't think the replication crisis is a reason to outright dismiss a paper, but certainly skepticism before replication is always warranted. I think one should also always be skeptical of how universal one's own experience truly is.
  - tomlockwood 4y ago
    I think I'd place this paper alone only a hair above someone's qualitative judgement, on the evidence pyramid, to be honest.
    - naasking 4y ago
      I don't know. It features thousands of data points from different groups in two different countries showing the same trend. That's substantially more compelling than anecdotes IMO, even after tempering expectations due to the replication crisis, but to each their own.
      - tomlockwood 4y ago
        The only reason we're seeing it is because a largely untrained audience decided to upvote it, because it matches their anecdotal experience enough that they think it's a good paper.
dTal 4y ago
As another random internet commenter, I anecdotally find that the least tolerant people I know are indeed substantially dumb. Thus, whatever probability you assign, between this comment and the original equally valid one contradicting it, we can derive no information, and must therefore rely on the study as the tiebreaker, however suspect we find it.
- Godel_unicode 4y ago
  Some of the (outwardly?) most tolerant people I know are quite dumb, and I know many extremely intelligent people capable of tremendous feats of abstract thought who are both very tolerant and very intolerant. This is an unreplicated social science paper. It’s worth approximately the paper it isn’t printed on.