4 ms·

I have known many clearly intelligent people who held irrational opinions quite strongly. For example, huge believers in specific, concrete stock market predict

I have known many clearly intelligent people who held irrational opinions quite strongly. For example, huge believers in specific, concrete stock market predictions based on some fibonacci sequence nonsense, to the point of actively loosing six figures through multiple failed predictions over a decade+ and still believing. So clearly, they're distinct. But the question about the bank teller doesn't seem like the strongest indicator. It seems more like a parlor trick that relies on our "system 1" brain short circuiting before our "system 2" kicks in. Better indicators would be ones that explore in-tribe vs. out-tribe perception of facts, or that attempted to quantify one's awareness of one's own blind spots.

thaumasiotes 10y ago
> the question about the bank teller doesn't seem like the strongest indicator. It seems more like a parlor trick that relies on our "system 1" brain short circuiting before our "system 2" kicks in. The question about the bank teller is a parlor trick that exploits normal language processing, specifically the Gricean Maxim of Relevance ( https://en.wikipedia.org/wiki/Cooperative_principle#Maxim_of_Relevance https://en.wikipedia.org/wiki/Cooperative_principle#Maxim_of... ). The maxim specifies that if somebody takes the trouble to point something out to you (say, that someone you've never met used to be a very politically-active left-winger), what they pointed out must be relevant to the conversation. It's basically another exercise in https://xkcd.com/169/ https://xkcd.com/169/ -- the experimenters aren't grasping the lesson that "communicating badly and then acting smug when you're misunderstood is not cleverness".
- nazgul17 10y ago
  This. I think the Linda problem is all based on the fact that when you tell people "which is more likely true ?" they hear "which one is more likely to be the exact description of reality?" Is not irrationality, just miscommunication.
  - Artoemius 10y ago
    The problem is that even if people hear it like this, P is still a more likely exact description of reality than P&Q.
    - Dylan16807 10y ago
      Nope! When your goal is an exact description, omitting information is just as bad as being wrong.
- Artoemius 10y ago
  > if somebody takes the trouble to point something out to you (say, that someone you've never met used to be a very politically-active left-winger), what they pointed out must be relevant to the conversation. Even if the brain believes that all things that are pointed out are relevant, that still doesn't explain this problem with bad communication. There are two statements, P and Q, and you have a question of what's more probable: statement P or the conjunction P&Q. The experimenter (communicating badly) makes you believe that Q has a very high probability. However, it is still logically impossible for P&Q to be more probable than P.
  - thaumasiotes 10y ago
    I relate the following discussion from memory: "Long ago", a psychologist did the following experiment on children of different ages: he'd lay out one line of M&Ms with broadly uniform spacing, and another line, longer, with the same spacing, and ask the child "which line has more M&Ms?" After they correctly indicated that the longer line had more M&Ms, he'd adjust the spacing between the M&Ms in the shorter line such that it was now longer. Then he'd ask again, "which line has more M&Ms?" He found that, below a certain age (I hope this finding was approximate), children would indicate the longer (and sparser) line as now having more M&Ms, and above the threshold age they would point to the correct, denser line. His published papers were viewed as establishing a major threshold in child development, where children started to be capable of assessing reality independently. "Not so long ago", someone decided to replicate this landmark finding. The twist was, in the replication, instead of asking the children "which line has more M&Ms now?", they said "OK, now you can take one of the lines and eat all of the M&Ms that were in it". And the threshold effect disappeared completely. Children of all ages chose the line with numerically more M&Ms, regardless of linear extent. When the problem is your experimental design, you're not warranted in drawing conclusions. You've already demonstrated that you're not qualified to speculate on them. > that still doesn't explain this problem with bad communication. > There are two statements, P and Q, and you have a question of what's more probable: statement P or the conjunction P&Q. The experimenter (communicating badly) makes you believe that Q has a very high probability. However, it is still logically impossible for P&Q to be more probable than P. Your objection only makes sense if (1) human-to-human communication consists of messages that precisely specify their intended meaning, and (2) people believe that (1) is true. (1) and (2) are both false.
- falcolas 10y ago
  FWIW, all human communication is this messy, so the ability to find rational answers even when confronted by this messiness has quite a bit of value. And if we can be trained to find rationality in the face of inexact human communication, all the better.
TheOtherHobbes 10y ago
>For example, huge believers in specific, concrete stock market predictions based on some fibonacci sequence nonsense, to the point of actively loosing six figures through multiple failed predictions over a decade+ and still believing. And there's the problem. We don't know what rationality or intelligence really are, but there's an assumption that they have something to do with modelling our current and future environment accurately and reliably. The subject is a mess of competing notions that aren't necessarily related. Does rationality mean being able to understand and manipulate abstract concepts and symbols using consistent rule sets ("market predictions", "Fibonacci") or does it mean making reliable winning predictions from limited information? Even the original feminist bank teller example is suspect. If you assume that the set of feminist bank tellers is smaller than the set of bank tellers, then the "rational" answer makes sense. But it's actually reasonable to read the question as "Given that one particular person is a bank teller - because both choices tell you she is - and has a history of activism, is she also likely to be a feminist?" One reading is purely mathematical, but the other makes perfect sense in a social context if you assume that you're talking about a real bank teller with a real history. If you were introduced to someone at a party and given that background information, which interpretation would be more likely to have predictive power?
- eli_gottlieb 10y ago
  Yep. Experiments in "rationality" and "irrationality" usually seem to rely on the experimenter rigging up an experimentally normative behavior (a "right answer") which differs, often dramatically, from the ecologically normative behavior (the correct way to act in the subjects' everyday lives). The experimenter then gets to publish an "interesting" paper showing that very clever people behave "wrongly" because they didn't leap to what were actually quite improbable interpretations of events (if you hadn't seen the experiment before). It makes it sound a lot like the Hollow-Face Illusion (https://en.wikipedia.org/wiki/Hollow-Face_illusion https://en.wikipedia.org/wiki/Hollow-Face_illusion), which we are subject to precisely because our expectations accurately reflect normal life. We need more experiments on rationality in which the "irrational" behavior is guaranteed not to be Bayes-optimal when given a previous human life.

thaumasiotes 10y ago

> the question about the bank teller doesn't seem like the strongest indicator. It seems more like a parlor trick that relies on our "system 1" brain short circuiting before our "system 2" kicks in. The question about the bank teller is a parlor trick that exploits normal language processing, specifically the Gricean Maxim of Relevance ( https://en.wikipedia.org/wiki/Cooperative_principle#Maxim_of_Relevance https://en.wikipedia.org/wiki/Cooperative_principle#Maxim_of... ). The maxim specifies that if somebody takes the trouble to point something out to you (say, that someone you've never met used to be a very politically-active left-winger), what they pointed out must be relevant to the conversation. It's basically another exercise in https://xkcd.com/169/ https://xkcd.com/169/ -- the experimenters aren't grasping the lesson that "communicating badly and then acting smug when you're misunderstood is not cleverness".

nazgul17 10y ago
This. I think the Linda problem is all based on the fact that when you tell people "which is more likely true ?" they hear "which one is more likely to be the exact description of reality?" Is not irrationality, just miscommunication.
- Artoemius 10y ago
  The problem is that even if people hear it like this, P is still a more likely exact description of reality than P&Q.
  - Dylan16807 10y ago
    Nope! When your goal is an exact description, omitting information is just as bad as being wrong.
Artoemius 10y ago
> if somebody takes the trouble to point something out to you (say, that someone you've never met used to be a very politically-active left-winger), what they pointed out must be relevant to the conversation. Even if the brain believes that all things that are pointed out are relevant, that still doesn't explain this problem with bad communication. There are two statements, P and Q, and you have a question of what's more probable: statement P or the conjunction P&Q. The experimenter (communicating badly) makes you believe that Q has a very high probability. However, it is still logically impossible for P&Q to be more probable than P.
- thaumasiotes 10y ago
  I relate the following discussion from memory: "Long ago", a psychologist did the following experiment on children of different ages: he'd lay out one line of M&Ms with broadly uniform spacing, and another line, longer, with the same spacing, and ask the child "which line has more M&Ms?" After they correctly indicated that the longer line had more M&Ms, he'd adjust the spacing between the M&Ms in the shorter line such that it was now longer. Then he'd ask again, "which line has more M&Ms?" He found that, below a certain age (I hope this finding was approximate), children would indicate the longer (and sparser) line as now having more M&Ms, and above the threshold age they would point to the correct, denser line. His published papers were viewed as establishing a major threshold in child development, where children started to be capable of assessing reality independently. "Not so long ago", someone decided to replicate this landmark finding. The twist was, in the replication, instead of asking the children "which line has more M&Ms now?", they said "OK, now you can take one of the lines and eat all of the M&Ms that were in it". And the threshold effect disappeared completely. Children of all ages chose the line with numerically more M&Ms, regardless of linear extent. When the problem is your experimental design, you're not warranted in drawing conclusions. You've already demonstrated that you're not qualified to speculate on them. > that still doesn't explain this problem with bad communication. > There are two statements, P and Q, and you have a question of what's more probable: statement P or the conjunction P&Q. The experimenter (communicating badly) makes you believe that Q has a very high probability. However, it is still logically impossible for P&Q to be more probable than P. Your objection only makes sense if (1) human-to-human communication consists of messages that precisely specify their intended meaning, and (2) people believe that (1) is true. (1) and (2) are both false.
falcolas 10y ago
FWIW, all human communication is this messy, so the ability to find rational answers even when confronted by this messiness has quite a bit of value. And if we can be trained to find rationality in the face of inexact human communication, all the better.

TheOtherHobbes 10y ago

>For example, huge believers in specific, concrete stock market predictions based on some fibonacci sequence nonsense, to the point of actively loosing six figures through multiple failed predictions over a decade+ and still believing. And there's the problem. We don't know what rationality or intelligence really are, but there's an assumption that they have something to do with modelling our current and future environment accurately and reliably. The subject is a mess of competing notions that aren't necessarily related. Does rationality mean being able to understand and manipulate abstract concepts and symbols using consistent rule sets ("market predictions", "Fibonacci") or does it mean making reliable winning predictions from limited information? Even the original feminist bank teller example is suspect. If you assume that the set of feminist bank tellers is smaller than the set of bank tellers, then the "rational" answer makes sense. But it's actually reasonable to read the question as "Given that one particular person is a bank teller - because both choices tell you she is - and has a history of activism, is she also likely to be a feminist?" One reading is purely mathematical, but the other makes perfect sense in a social context if you assume that you're talking about a real bank teller with a real history. If you were introduced to someone at a party and given that background information, which interpretation would be more likely to have predictive power?

eli_gottlieb 10y ago
Yep. Experiments in "rationality" and "irrationality" usually seem to rely on the experimenter rigging up an experimentally normative behavior (a "right answer") which differs, often dramatically, from the ecologically normative behavior (the correct way to act in the subjects' everyday lives). The experimenter then gets to publish an "interesting" paper showing that very clever people behave "wrongly" because they didn't leap to what were actually quite improbable interpretations of events (if you hadn't seen the experiment before). It makes it sound a lot like the Hollow-Face Illusion (https://en.wikipedia.org/wiki/Hollow-Face_illusion https://en.wikipedia.org/wiki/Hollow-Face_illusion), which we are subject to precisely because our expectations accurately reflect normal life. We need more experiments on rationality in which the "irrational" behavior is guaranteed not to be Bayes-optimal when given a previous human life.