5 ms·

I'm just done with a 3-hour reading session of an evolutionary psychology book by one of the leading scientists in the field. The book is extremely competently

I'm just done with a 3-hour reading session of an evolutionary psychology book by one of the leading scientists in the field. The book is extremely competently written, and is awash with statistics on almost every page, "70% of men this; 30% of women that ... on and on". And much to my solace, the scientist was super-careful to distinguish studies that were replicable, and those that were not. Still, reading your comment makes me despair. It plants a nagging doubt in my mind, "how many of these zillion studies cited that are actually replicable?" This doubt remains despite knowing that the scientist is one of the leading experts in the field, and very down-to-earth. What are the solutions here? A big incentive-shift to reward replication more? Public shaming of misleading studies? Influential conferences giving more air-time for talks about "studies that did not replicate"? I know some of these happen at a smaller-scale[1], but I wonder about the "scaling" aspect (to use a very HN-esque term). PS: Since I read Behave by Sapolsky — where he says "your prefrontal cortex [which plays critical role in cognition, emotional regulation, and control of impulsive behavior] doesn't come online until you are 24" — I tend to take all studies done on university campuses with students younger than 24 with a good spoon of salt. ;-) [1] https://replicationindex.com/about/ https://replicationindex.com/about/

api 5y ago
Evo psych is questionable to me for more basic reasons. It seems full of untestable just so stories to explain apparent biases that are themselves hard to pin down or prove are a result of nature not nurture. It’s probably not all bullshit but I would bet a double digit percentage of it is.
- candiodari 5y ago
  psych is questionable for basic reasons. It is a humanities science. It's purpose is not to figure out the world but to change it. Figure out how to end poverty for example. Therefore it is not well suited to figure out the world. You should treat all of it with extreme helpings of salt.
  - hexane360 5y ago
    Can't this be applied to wide swaths of hard sciences as well? Lots of scientific work overlaps heavily with engineering, which is all about changing the world. Also, I don't think ending poverty is a major stated goal of psychology research. . .
- kashyapc 5y ago
  I'm conscious that this is a flame-bait topic. That said, no, dismissing the whole field as "questionable" is callous. Yes, there are many open questions, loaded landmines, and ethical concerns in evolutionary psychology research. But there's also copious evidence in its favour. (Reference: David Buss et al.) Many people might spare themselves at least some misery by educating themselves about evolutionary psychology, including the landmines and open questions.
dahart 5y ago
> “how many of these zillion studies cited that are actually replicable?” This doubt remains despite knowing that the scientist is one of the leading experts in the field, and very down-to-earth. I think the problem is much bigger than simply a binary is it replicable or not. It’s extremely easy to find papers by “leading experts” that have valid data with replicable results where the conclusions have been generalized beyond the experiments. The media does this more or less by default when reporting on scientific results, but researchers do it themselves to a huge degree, use very specific conditions and results to jump to a wider conclusion that is not actually supported by the results. A high profile example of this is the “Dunning Kruger” effect; the data in paper did not show what the flowery narrative in the paper claimed to show, but there’s no reason to think they falsified the results. Some researchers have reproduced the results, as long as the conditions were very similar. Other researchers have tried to reproduce the results under different conditions that should have worked according to the paper’s narrative and conclusions, but found that they could not reproduce, because there were specific factors in the original experiment that were not discussed in the original paper’s conclusions -- in other words, Dunning and Kruger overstated what they measured such that the conclusion was not true. They both enjoyed successful academic careers and some degree of academic fame as a result of this paper that is technically reproducible but not generally true. To make matters worse, the public has generally misinterpreted and misunderstood even the incorrect conclusions the authors stated, and turned it into something else. Almost never in discussions where the DK effect is invoked do people talk about the context or methodology of the experiments, or the people who participated in them. This human tendency to tell a story and lose the context and details and specificity of the original evidence, the tendency to declare that one piece of evidence means there is a general truth, that is scarier to me than whether papers are replicable or not, because it casts doubt on all the replicable papers too.
- kashyapc 5y ago
  I fully agree. Thanks for the excellent articulation of the layered complexity involved here, including a chilling example.
mistermann 5y ago
One approach that can be adopted on a personal level is simply changing the way one thinks. For example, switch from a binary (true/false) method of epistemology to trinary (true/false/unknown), defaulting to unknown, and consciously insist on a high level of certainty to reclassify an idea. There's obviously more complexity than this, but I believe that if even a relatively small percentage of the population started thinking like this (particularly, influential people) it could make a very big difference. Unfortunately, this seems to be extremely counter to human nature and desires - people seem seem compelled to form conclusions, even when it is not necessary ("Do people have ideas, or do ideas have people?").
pkkm 5y ago
> The book is extremely competently written, and is awash with statistics on almost every page, "70% of men this; 30% of women that ... on and on". And much to my solace, the scientist was super-careful to distinguish studies that were replicable, and those that were not. Out of curiosity, what's the title of the book?
- kashyapc 5y ago
  Evolutionary Psychology by David Buss[1]. [1] https://www.routledge.com/Evolutionary-Psychology-The-New-Science-of-the-Mind/Buss/p/book/9781138088610 https://www.routledge.com/Evolutionary-Psychology-The-New-Sc...

api 5y ago

Evo psych is questionable to me for more basic reasons. It seems full of untestable just so stories to explain apparent biases that are themselves hard to pin down or prove are a result of nature not nurture. It’s probably not all bullshit but I would bet a double digit percentage of it is.

candiodari 5y ago
psych is questionable for basic reasons. It is a humanities science. It's purpose is not to figure out the world but to change it. Figure out how to end poverty for example. Therefore it is not well suited to figure out the world. You should treat all of it with extreme helpings of salt.
- hexane360 5y ago
  Can't this be applied to wide swaths of hard sciences as well? Lots of scientific work overlaps heavily with engineering, which is all about changing the world. Also, I don't think ending poverty is a major stated goal of psychology research. . .
kashyapc 5y ago
I'm conscious that this is a flame-bait topic. That said, no, dismissing the whole field as "questionable" is callous. Yes, there are many open questions, loaded landmines, and ethical concerns in evolutionary psychology research. But there's also copious evidence in its favour. (Reference: David Buss et al.) Many people might spare themselves at least some misery by educating themselves about evolutionary psychology, including the landmines and open questions.

dahart 5y ago

> “how many of these zillion studies cited that are actually replicable?” This doubt remains despite knowing that the scientist is one of the leading experts in the field, and very down-to-earth. I think the problem is much bigger than simply a binary is it replicable or not. It’s extremely easy to find papers by “leading experts” that have valid data with replicable results where the conclusions have been generalized beyond the experiments. The media does this more or less by default when reporting on scientific results, but researchers do it themselves to a huge degree, use very specific conditions and results to jump to a wider conclusion that is not actually supported by the results. A high profile example of this is the “Dunning Kruger” effect; the data in paper did not show what the flowery narrative in the paper claimed to show, but there’s no reason to think they falsified the results. Some researchers have reproduced the results, as long as the conditions were very similar. Other researchers have tried to reproduce the results under different conditions that should have worked according to the paper’s narrative and conclusions, but found that they could not reproduce, because there were specific factors in the original experiment that were not discussed in the original paper’s conclusions -- in other words, Dunning and Kruger overstated what they measured such that the conclusion was not true. They both enjoyed successful academic careers and some degree of academic fame as a result of this paper that is technically reproducible but not generally true. To make matters worse, the public has generally misinterpreted and misunderstood even the incorrect conclusions the authors stated, and turned it into something else. Almost never in discussions where the DK effect is invoked do people talk about the context or methodology of the experiments, or the people who participated in them. This human tendency to tell a story and lose the context and details and specificity of the original evidence, the tendency to declare that one piece of evidence means there is a general truth, that is scarier to me than whether papers are replicable or not, because it casts doubt on all the replicable papers too.

kashyapc 5y ago
I fully agree. Thanks for the excellent articulation of the layered complexity involved here, including a chilling example.

mistermann 5y ago

One approach that can be adopted on a personal level is simply changing the way one thinks. For example, switch from a binary (true/false) method of epistemology to trinary (true/false/unknown), defaulting to unknown, and consciously insist on a high level of certainty to reclassify an idea. There's obviously more complexity than this, but I believe that if even a relatively small percentage of the population started thinking like this (particularly, influential people) it could make a very big difference. Unfortunately, this seems to be extremely counter to human nature and desires - people seem seem compelled to form conclusions, even when it is not necessary ("Do people have ideas, or do ideas have people?").

pkkm 5y ago

> The book is extremely competently written, and is awash with statistics on almost every page, "70% of men this; 30% of women that ... on and on". And much to my solace, the scientist was super-careful to distinguish studies that were replicable, and those that were not. Out of curiosity, what's the title of the book?

kashyapc 5y ago
Evolutionary Psychology by David Buss[1]. [1] https://www.routledge.com/Evolutionary-Psychology-The-New-Science-of-the-Mind/Buss/p/book/9781138088610 https://www.routledge.com/Evolutionary-Psychology-The-New-Sc...