3 ms·

I'd again mention the ongoing replication crisis in psychology. People have specifically attempted to replicate Paul Piff's numerous studies trying to assign v

by rjf72 7y ago

I'd again mention the ongoing replication crisis in psychology. People have specifically attempted to replicate Paul Piff's numerous studies trying to assign various negative characteristics to those of upper social class. The results did find some correlations maintained, such as those between greed and unethical behavior, but Piff's keystone correlation between social class and unethical behavior was completely unable to be replicated. I do not think people understand how bad of shape psychology is in right now, social psychology in particular. Social psychology replication studies tend to be hitting around 20%. For instance once again the Journal of Personality and Social Psychology comes up. It's interestingly enough the journal that published the study that this link is discussing, as well as the journal that Piff published in. Its current replication rate is 23%. [2] [1] - https://www.collabra.org/article/10.1525/collabra.166/ https://www.collabra.org/article/10.1525/collabra.166/ [2] - https://en.wikipedia.org/wiki/Replication_crisis#Psychology_replication_rates https://en.wikipedia.org/wiki/Replication_crisis#Psychology_...

gerbilly 7y ago
> I do not think people understand how bad of shape psychology is in right now, social psychology in particular. Social psychology replication studies tend to be hitting around 20%. Yes I think we do, at least on HN, because anytime any psychology study or result is mentioned at least two threads popup up citing the replication crisis. The thing is, it kind of depends on how you define replication, also some of these studies were done decades ago, society has changed since then, possibly affecting the results. How easy would it be to replicate the bystander apathy studies where people were in a smoke filled room after 9/11, for example?¹ It definitely should be taken into account, but it's much easier to tear down than to give a critical appraisal. I don't mean to criticize your comment in particular, because it was actually a good appraisal of this particular body of work. I do think HN has some biases² and the tendency to shout "replication crisis" in a self satisfied manner is one of the things that bug me the most about HN.³ 1: https://psycnet.apa.org/record/1969-03938-001 https://psycnet.apa.org/record/1969-03938-001 2: No politics, but in human affairs, everything is politics. 3: Especially since most of us on here are from the hard sciences, which ironically are easier because they deal in concrete, measurable phenomena.
- TeMPOraL 7y ago
  > The thing is, it kind of depends on how you define replication, also some of these studies were done decades ago, society has changed since then, possibly affecting the results. If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society. It belongs to historical studies at best, at worst it was garbage from the get-go (I suspect the latter is usually the case; human psychology seems mostly fixed). Yes, hard sciences are easier; the physical reality keeps you honest. But that only means soft sciences need even stricter standards than hard ones, whereas today the opposite is the case. There's little point in spending all this effort if almost all of it is wasted - we need to accept that psychology is hard, and meaningful results will be rare.
  - gerbilly 7y ago
    > If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society. Not necessarily, again it depends what you mean by replication. For example in the case of the bystander apathy experiment where they used smoke, do you have to replicate the study exactly as it was done, i.e.: is smoke essential, or would some other sign of threat also do? This is why I chose that example, if right after 9/11 you tried to replicate that particular study in the US and used smoke, it probably wouldn't replicate, because people would have been recently and strongly primed to react to smoke as a threat. I think people are over-interpreting the replication failures sometimes.
    - TeMPOraL 7y ago
      9/11 was an exceptional circumstance. You could try and replicate that study again, some time after it - e.g. now. If the study still won't replicate, that's evidence for it being bogus. Single replication failure isn't necessarily damning for a study, much less the entire field. But if most studies keep on not replicating, then you have a problem.
- bloaf 7y ago
  One other thing to keep in mind about social psychology: http://www.spsp.org/news-center/blog/stereotype-accuracy-response http://www.spsp.org/news-center/blog/stereotype-accuracy-res... > Stereotype accuracy is one of the largest and most replicable effects in all of social psychology. Richard et al (2003) found that fewer than 5% of all effects in social psychology exceeded r’s of .50. In contrast, nearly all consensual stereotype accuracy correlations and about half of all personal stereotype accuracy correlations exceed .50. That is, stereotypes are frequently better predictors of human behavior than bona-fide research in social psychology.

by rjf72 7y ago

gerbilly 7y ago

> I do not think people understand how bad of shape psychology is in right now, social psychology in particular. Social psychology replication studies tend to be hitting around 20%. Yes I think we do, at least on HN, because anytime any psychology study or result is mentioned at least two threads popup up citing the replication crisis. The thing is, it kind of depends on how you define replication, also some of these studies were done decades ago, society has changed since then, possibly affecting the results. How easy would it be to replicate the bystander apathy studies where people were in a smoke filled room after 9/11, for example?¹ It definitely should be taken into account, but it's much easier to tear down than to give a critical appraisal. I don't mean to criticize your comment in particular, because it was actually a good appraisal of this particular body of work. I do think HN has some biases² and the tendency to shout "replication crisis" in a self satisfied manner is one of the things that bug me the most about HN.³ 1: https://psycnet.apa.org/record/1969-03938-001 https://psycnet.apa.org/record/1969-03938-001 2: No politics, but in human affairs, everything is politics. 3: Especially since most of us on here are from the hard sciences, which ironically are easier because they deal in concrete, measurable phenomena.

TeMPOraL 7y ago
> The thing is, it kind of depends on how you define replication, also some of these studies were done decades ago, society has changed since then, possibly affecting the results. If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society. It belongs to historical studies at best, at worst it was garbage from the get-go (I suspect the latter is usually the case; human psychology seems mostly fixed). Yes, hard sciences are easier; the physical reality keeps you honest. But that only means soft sciences need even stricter standards than hard ones, whereas today the opposite is the case. There's little point in spending all this effort if almost all of it is wasted - we need to accept that psychology is hard, and meaningful results will be rare.
- gerbilly 7y ago
  > If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society. Not necessarily, again it depends what you mean by replication. For example in the case of the bystander apathy experiment where they used smoke, do you have to replicate the study exactly as it was done, i.e.: is smoke essential, or would some other sign of threat also do? This is why I chose that example, if right after 9/11 you tried to replicate that particular study in the US and used smoke, it probably wouldn't replicate, because people would have been recently and strongly primed to react to smoke as a threat. I think people are over-interpreting the replication failures sometimes.
  - TeMPOraL 7y ago
    9/11 was an exceptional circumstance. You could try and replicate that study again, some time after it - e.g. now. If the study still won't replicate, that's evidence for it being bogus. Single replication failure isn't necessarily damning for a study, much less the entire field. But if most studies keep on not replicating, then you have a problem.
bloaf 7y ago
One other thing to keep in mind about social psychology: http://www.spsp.org/news-center/blog/stereotype-accuracy-response http://www.spsp.org/news-center/blog/stereotype-accuracy-res... > Stereotype accuracy is one of the largest and most replicable effects in all of social psychology. Richard et al (2003) found that fewer than 5% of all effects in social psychology exceeded r’s of .50. In contrast, nearly all consensual stereotype accuracy correlations and about half of all personal stereotype accuracy correlations exceed .50. That is, stereotypes are frequently better predictors of human behavior than bona-fide research in social psychology.