3 ms·

Are these two limitations of rational thought described in the literature: 1. Rational thought takes place in an irrational environment. As much as we try to c

by csours 2y ago

Are these two limitations of rational thought described in the literature: 1. Rational thought takes place in an irrational environment. As much as we try to control for biases, fallacies, heuristics, effects, etc, our evaluation of a scenario is dependent upon our personal history. 2. Rationality is a great tool for breaking things into small parts and describing acting in small steps; but a fundamental error occurs when we 'zoom out' to a system and use those small parts to say something definitive about the whole. In each step 'down', we exclude details not relevant to that scope, but we neglect to add those details back when we consider the system. AI will certainly not save us from #1 because it will have it's own history. #2 may benefit from AI, but the people using it will still have great difficulty.

mistermann 2y ago
I have no idea about literature, but off the cuff I would say: > 1. Rational thought takes place in an irrational environment. As much as we try to control for biases, fallacies, heuristics, effects, etc, our evaluation of a scenario is dependent upon our personal history. a) rational thought (usually) isn't what it seems to be b) this is sloppy/inappropriate/dangerous language for this topic > Rationality is a great tool for breaking things into small parts and describing acting in small steps; but a fundamental error occurs when we 'zoom out' to a system and use those small parts to say something definitive about the whole. Statistically/probabilistically, yes (take this comment section, on a high-IQ platform (see also: /r/SlateStarCodex), as a prime example), but not necessarily. Doing it wrong will produce wrong results, like most anything else. An important distinction though in some domains: it is not possible to realize when you are wrong. And sometimes, it isn't even possible to wonder if you are wrong[1]. Consider the complexity involved (and the historic journey behind) manufacturing modern day top end CPU's - and this is but one of the many feats of genius humanity has pulled off, and we now take much of it for granted. Why can we do that, but not certain classes of other cognitively complicated tasks? What is the distinction between the domains that guarantees (so far) failure? (A hint: consider how we answer such questions.) > #2 may benefit from AI, but the people using it will still have great difficulty. What is the meaning of the symbol "great" in this context? Is it related to the probability of us Accomplishing The Goal? ------------------ [1] A classic "extraordinary claim", typically invoking this (and then after that, something else....but discussing those sorts of things isn't something you do in polite company, which HN most certainly is[2]): https://en.wikipedia.org/wiki/Extraordinary_claims_require_extraordinary_evidence https://en.wikipedia.org/wiki/Extraordinary_claims_require_e... A knockout punch if I've ever seen one! [2] https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- csours 2y ago
  > b) this is sloppy/inappropriate/dangerous language for this topic Yes, I'm describing Untrue Scotsmen. I'm hoping for a citation or something I can google. People are farting in my rational cleanroom and I don't like it.
  - mistermann 2y ago
    > I'm hoping for a citation or something I can google. Not entirely sure what you mean, something like this? https://plato.stanford.edu/entries/knowledge-analysis https://plato.stanford.edu/entries/knowledge-analysis https://en.wikipedia.org/wiki/Direct_and_indirect_realism https://en.wikipedia.org/wiki/Direct_and_indirect_realism https://en.wikipedia.org/wiki/Tautology_(logic) https://en.wikipedia.org/wiki/Tautology_(logic)

by csours 2y ago

mistermann 2y ago

I have no idea about literature, but off the cuff I would say: > 1. Rational thought takes place in an irrational environment. As much as we try to control for biases, fallacies, heuristics, effects, etc, our evaluation of a scenario is dependent upon our personal history. a) rational thought (usually) isn't what it seems to be b) this is sloppy/inappropriate/dangerous language for this topic > Rationality is a great tool for breaking things into small parts and describing acting in small steps; but a fundamental error occurs when we 'zoom out' to a system and use those small parts to say something definitive about the whole. Statistically/probabilistically, yes (take this comment section, on a high-IQ platform (see also: /r/SlateStarCodex), as a prime example), but not necessarily. Doing it wrong will produce wrong results, like most anything else. An important distinction though in some domains: it is not possible to realize when you are wrong. And sometimes, it isn't even possible to wonder if you are wrong[1]. Consider the complexity involved (and the historic journey behind) manufacturing modern day top end CPU's - and this is but one of the many feats of genius humanity has pulled off, and we now take much of it for granted. Why can we do that, but not certain classes of other cognitively complicated tasks? What is the distinction between the domains that guarantees (so far) failure? (A hint: consider how we answer such questions.) > #2 may benefit from AI, but the people using it will still have great difficulty. What is the meaning of the symbol "great" in this context? Is it related to the probability of us Accomplishing The Goal? ------------------ [1] A classic "extraordinary claim", typically invoking this (and then after that, something else....but discussing those sorts of things isn't something you do in polite company, which HN most certainly is[2]): https://en.wikipedia.org/wiki/Extraordinary_claims_require_extraordinary_evidence https://en.wikipedia.org/wiki/Extraordinary_claims_require_e... A knockout punch if I've ever seen one! [2] https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

csours 2y ago
> b) this is sloppy/inappropriate/dangerous language for this topic Yes, I'm describing Untrue Scotsmen. I'm hoping for a citation or something I can google. People are farting in my rational cleanroom and I don't like it.
- mistermann 2y ago
  > I'm hoping for a citation or something I can google. Not entirely sure what you mean, something like this? https://plato.stanford.edu/entries/knowledge-analysis https://plato.stanford.edu/entries/knowledge-analysis https://en.wikipedia.org/wiki/Direct_and_indirect_realism https://en.wikipedia.org/wiki/Direct_and_indirect_realism https://en.wikipedia.org/wiki/Tautology_(logic) https://en.wikipedia.org/wiki/Tautology_(logic)