4 ms·

> I do not think people understand how bad of shape psychology is in right now, social psychology in particular. Social psychology replication studies tend to b

> I do not think people understand how bad of shape psychology is in right now, social psychology in particular. Social psychology replication studies tend to be hitting around 20%. Yes I think we do, at least on HN, because anytime any psychology study or result is mentioned at least two threads popup up citing the replication crisis. The thing is, it kind of depends on how you define replication, also some of these studies were done decades ago, society has changed since then, possibly affecting the results. How easy would it be to replicate the bystander apathy studies where people were in a smoke filled room after 9/11, for example?¹ It definitely should be taken into account, but it's much easier to tear down than to give a critical appraisal. I don't mean to criticize your comment in particular, because it was actually a good appraisal of this particular body of work. I do think HN has some biases² and the tendency to shout "replication crisis" in a self satisfied manner is one of the things that bug me the most about HN.³ 1: https://psycnet.apa.org/record/1969-03938-001 https://psycnet.apa.org/record/1969-03938-001 2: No politics, but in human affairs, everything is politics. 3: Especially since most of us on here are from the hard sciences, which ironically are easier because they deal in concrete, measurable phenomena.

TeMPOraL 7y ago
> The thing is, it kind of depends on how you define replication, also some of these studies were done decades ago, society has changed since then, possibly affecting the results. If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society. It belongs to historical studies at best, at worst it was garbage from the get-go (I suspect the latter is usually the case; human psychology seems mostly fixed). Yes, hard sciences are easier; the physical reality keeps you honest. But that only means soft sciences need even stricter standards than hard ones, whereas today the opposite is the case. There's little point in spending all this effort if almost all of it is wasted - we need to accept that psychology is hard, and meaningful results will be rare.
- gerbilly 7y ago
  > If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society. Not necessarily, again it depends what you mean by replication. For example in the case of the bystander apathy experiment where they used smoke, do you have to replicate the study exactly as it was done, i.e.: is smoke essential, or would some other sign of threat also do? This is why I chose that example, if right after 9/11 you tried to replicate that particular study in the US and used smoke, it probably wouldn't replicate, because people would have been recently and strongly primed to react to smoke as a threat. I think people are over-interpreting the replication failures sometimes.
  - TeMPOraL 7y ago
    9/11 was an exceptional circumstance. You could try and replicate that study again, some time after it - e.g. now. If the study still won't replicate, that's evidence for it being bogus. Single replication failure isn't necessarily damning for a study, much less the entire field. But if most studies keep on not replicating, then you have a problem.
bloaf 7y ago
One other thing to keep in mind about social psychology: http://www.spsp.org/news-center/blog/stereotype-accuracy-response http://www.spsp.org/news-center/blog/stereotype-accuracy-res... > Stereotype accuracy is one of the largest and most replicable effects in all of social psychology. Richard et al (2003) found that fewer than 5% of all effects in social psychology exceeded r’s of .50. In contrast, nearly all consensual stereotype accuracy correlations and about half of all personal stereotype accuracy correlations exceed .50. That is, stereotypes are frequently better predictors of human behavior than bona-fide research in social psychology.
- deleted 7y ago
  [deleted]
cam_l 7y ago
>hard sciences Yes, i have often said (including here on HN) that we should refer to them as the easy sciences, or less incendiary, the concrete sciences. The question that lingers in my mind about the replication issue is perhaps the empirical process itself is ill suited to psychology. I think the replication crisis might be a crisis of process as much as it is of incentives. Empiricism seeks repeatable phenomena, same inputs, same outputs. In our psychological, virtual worlds, we have different inputs, different outputs, but are looking for similar processes. (Your example is a good one, in that the underlying process is a common experience but the lack of replicability of the study does not invalidate the process, just the input-ouput.) Maybe we need another model, or another layer, of empiricism in psychology.. I would characterise it as a sort of computational model. I am reminded of the study where the neurologists try to reverse engineer a computer program by looking at a computer.[0] I can't help but wonder how psychologists would approach the same problem. [0] cannot find it at the moment, but admittedly do like the analogy in that it is suggestive of the difficulty of understanding a complex system outside of its abstraction layer.
- username90 7y ago
  > Yes, i have often said (including here on HN) that we should refer to them as the easy sciences, or less incendiary, the concrete sciences. They are hard as in unyielding, not hard as in not easy. So hard sciences are hard because their results are easy to replicate.
  - cam_l 7y ago
    English has many terms with multiple meanings which we use with the express intent to imbue our words with adjacent meaning. A hard decision is difficult but also unyielding. A hard man is unyielding but also strong. A strong decision is controversial but also tough. These dual meanings are often so ingrained we cannot see the inherent issues with their usage. They are often somewhat Orwellian in their duality. The hard sciences is one of those terms. Hence my joke about the easy sciences.

TeMPOraL 7y ago

> The thing is, it kind of depends on how you define replication, also some of these studies were done decades ago, society has changed since then, possibly affecting the results. If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society. It belongs to historical studies at best, at worst it was garbage from the get-go (I suspect the latter is usually the case; human psychology seems mostly fixed). Yes, hard sciences are easier; the physical reality keeps you honest. But that only means soft sciences need even stricter standards than hard ones, whereas today the opposite is the case. There's little point in spending all this effort if almost all of it is wasted - we need to accept that psychology is hard, and meaningful results will be rare.

gerbilly 7y ago
> If a study doesn't replicate because society has changed, and so the results, then that study cannot be used to infer things about current society. Not necessarily, again it depends what you mean by replication. For example in the case of the bystander apathy experiment where they used smoke, do you have to replicate the study exactly as it was done, i.e.: is smoke essential, or would some other sign of threat also do? This is why I chose that example, if right after 9/11 you tried to replicate that particular study in the US and used smoke, it probably wouldn't replicate, because people would have been recently and strongly primed to react to smoke as a threat. I think people are over-interpreting the replication failures sometimes.
- TeMPOraL 7y ago
  9/11 was an exceptional circumstance. You could try and replicate that study again, some time after it - e.g. now. If the study still won't replicate, that's evidence for it being bogus. Single replication failure isn't necessarily damning for a study, much less the entire field. But if most studies keep on not replicating, then you have a problem.

bloaf 7y ago

One other thing to keep in mind about social psychology: http://www.spsp.org/news-center/blog/stereotype-accuracy-response http://www.spsp.org/news-center/blog/stereotype-accuracy-res... > Stereotype accuracy is one of the largest and most replicable effects in all of social psychology. Richard et al (2003) found that fewer than 5% of all effects in social psychology exceeded r’s of .50. In contrast, nearly all consensual stereotype accuracy correlations and about half of all personal stereotype accuracy correlations exceed .50. That is, stereotypes are frequently better predictors of human behavior than bona-fide research in social psychology.

deleted 7y ago
[deleted]

cam_l 7y ago

>hard sciences Yes, i have often said (including here on HN) that we should refer to them as the easy sciences, or less incendiary, the concrete sciences. The question that lingers in my mind about the replication issue is perhaps the empirical process itself is ill suited to psychology. I think the replication crisis might be a crisis of process as much as it is of incentives. Empiricism seeks repeatable phenomena, same inputs, same outputs. In our psychological, virtual worlds, we have different inputs, different outputs, but are looking for similar processes. (Your example is a good one, in that the underlying process is a common experience but the lack of replicability of the study does not invalidate the process, just the input-ouput.) Maybe we need another model, or another layer, of empiricism in psychology.. I would characterise it as a sort of computational model. I am reminded of the study where the neurologists try to reverse engineer a computer program by looking at a computer.[0] I can't help but wonder how psychologists would approach the same problem. [0] cannot find it at the moment, but admittedly do like the analogy in that it is suggestive of the difficulty of understanding a complex system outside of its abstraction layer.

username90 7y ago
> Yes, i have often said (including here on HN) that we should refer to them as the easy sciences, or less incendiary, the concrete sciences. They are hard as in unyielding, not hard as in not easy. So hard sciences are hard because their results are easy to replicate.
- cam_l 7y ago
  English has many terms with multiple meanings which we use with the express intent to imbue our words with adjacent meaning. A hard decision is difficult but also unyielding. A hard man is unyielding but also strong. A strong decision is controversial but also tough. These dual meanings are often so ingrained we cannot see the inherent issues with their usage. They are often somewhat Orwellian in their duality. The hard sciences is one of those terms. Hence my joke about the easy sciences.