3 ms·

The studies in Thinking Fast and Slow mostly failed replication: https://replicationindex.com/2020/12/30/a-meta-scientific-perspective-on-thinking-fast-and-slo

The studies in Thinking Fast and Slow mostly failed replication: https://replicationindex.com/2020/12/30/a-meta-scientific-perspective-on-thinking-fast-and-slow/ https://replicationindex.com/2020/12/30/a-meta-scientific-pe...

glial 1y ago
Yes, but the overall framework - that our thinking can be broadly divided into habits/reaction and deliberation/forecasting - still holds, with a ton of supporting neural evidence.
- an0malous 1y ago
  I don’t think even that is true or particularly useful. Why does it need to be binary? Couldn’t it be a spectrum? Are “fast thoughts” always habits and reactions? Some people can calculate large numbers very quickly, some people describe their inspiration for complex ideas happening in a single moment like a muse delivered it to them. Some people process thoughts and feelings for decades until they suddenly snap. I don’t think this “fast vs slow thoughts” framework really has any value to it, and it excludes a lot of thought patterns.
  - glial 1y ago
    In terms of being useful, it depends on the field. In reinforcement learning, for example, the entire field is divided into 'model-free' and 'model-based' approaches. Model-free approaches are learning state-action mappings, which are like habits or 'fast' thinking. Model-based approaches try to build a world model that allows planning and forecasting. If you're looking for utility in terms of understanding cognition, here are some resources if you're interested - pop-sci books written by cognitive scientists: - https://global.oup.com/academic/product/the-mind-within-the-brain-9780199891887?cc=us&lang=en& https://global.oup.com/academic/product/the-mind-within-the-... - https://press.uchicago.edu/ucp/books/book/chicago/R/bo3627517.html https://press.uchicago.edu/ucp/books/book/chicago/R/bo362751... Is the division true? Of course not - to paraphrase Box, all models are wrong. But some are useful. This one is useful.
Recursing 1y ago
That's not true, or at least it only applies to some chapters. Most studies in the book did replicate.
- username332211 1y ago
  Do you happen to know how many of the interesting studies replicated? A book can cite of studies, but the principal conclusions could be based only on a small subset of the citations. I particular, I'd be very curios to know how many of the replicable results require a "behavioral" explanation and how many are explainable trough the utility functions and rational agents.
- an0malous 1y ago
  No they did not, from the link I shared: > The mean power is 46%, which implies that only half of the results would be replicated in exact replication studies. The success rate in actual replication studies is often lower and may be as low as the estimated discovery rate (Bartos & Schimmack, 2020). So, replicability is somewhere between 12% and 46%. Even if half of the results are replicable, we do not know which results are replicable and which one’s are not. The Chapter-based analyses provide some clues which findings may be less trustworthy (implicit priming) and which ones may be more trustworthy (overconfidence), but the main conclusion is that the empirical basis for claims in “Thinking: Fast and Slow” is shaky.

glial 1y ago

Yes, but the overall framework - that our thinking can be broadly divided into habits/reaction and deliberation/forecasting - still holds, with a ton of supporting neural evidence.

an0malous 1y ago
I don’t think even that is true or particularly useful. Why does it need to be binary? Couldn’t it be a spectrum? Are “fast thoughts” always habits and reactions? Some people can calculate large numbers very quickly, some people describe their inspiration for complex ideas happening in a single moment like a muse delivered it to them. Some people process thoughts and feelings for decades until they suddenly snap. I don’t think this “fast vs slow thoughts” framework really has any value to it, and it excludes a lot of thought patterns.
- glial 1y ago
  In terms of being useful, it depends on the field. In reinforcement learning, for example, the entire field is divided into 'model-free' and 'model-based' approaches. Model-free approaches are learning state-action mappings, which are like habits or 'fast' thinking. Model-based approaches try to build a world model that allows planning and forecasting. If you're looking for utility in terms of understanding cognition, here are some resources if you're interested - pop-sci books written by cognitive scientists: - https://global.oup.com/academic/product/the-mind-within-the-brain-9780199891887?cc=us&lang=en& https://global.oup.com/academic/product/the-mind-within-the-... - https://press.uchicago.edu/ucp/books/book/chicago/R/bo3627517.html https://press.uchicago.edu/ucp/books/book/chicago/R/bo362751... Is the division true? Of course not - to paraphrase Box, all models are wrong. But some are useful. This one is useful.

Recursing 1y ago

That's not true, or at least it only applies to some chapters. Most studies in the book did replicate.

username332211 1y ago
Do you happen to know how many of the interesting studies replicated? A book can cite of studies, but the principal conclusions could be based only on a small subset of the citations. I particular, I'd be very curios to know how many of the replicable results require a "behavioral" explanation and how many are explainable trough the utility functions and rational agents.
an0malous 1y ago
No they did not, from the link I shared: > The mean power is 46%, which implies that only half of the results would be replicated in exact replication studies. The success rate in actual replication studies is often lower and may be as low as the estimated discovery rate (Bartos & Schimmack, 2020). So, replicability is somewhere between 12% and 46%. Even if half of the results are replicable, we do not know which results are replicable and which one’s are not. The Chapter-based analyses provide some clues which findings may be less trustworthy (implicit priming) and which ones may be more trustworthy (overconfidence), but the main conclusion is that the empirical basis for claims in “Thinking: Fast and Slow” is shaky.