11 ms·

I maintain that it isn't just hard, it is computationally impossible. We should all know that given a belief about the world, and evidence, Bayes' Theorem desc

by btilly 5y ago

I maintain that it isn't just hard, it is computationally impossible. We should all know that given a belief about the world, and evidence, Bayes' Theorem describes how to update our beliefs. But what if we have a network of interrelated beliefs? That's called a Bayesian net, and it turns out that Bayes' Theorem also prescribes a unique answer. However, unfortunately, it turns out that working out that answer is NP-hard. OK, you say, we can come up with an approximate answer. Sorry, no, coming up with an approximate answer that gets within probability 0.5 - ε, for 0 < ε, is ALSO NP-hard. It is literally true that under the right circumstances a single data point logically should be able to flip our entire world view, and which data point does it is computationally intractible. Therefore our brains use a bunch of heuristics, with a bunch of known failure modes. You can read all the lesswrong you want. You can read Thinking, Fast and Slow and learn why we fail as we do. But the one thing that we cannot do, no matter how much work or effort we put into it, is have the sheer brainpower required to actually BE rational. The effort of doing better is still worthwhile. But the goal itself is unachievable.

kirso 5y ago
Does this mean then that all the mental models are basically useless because the outcome may be irrational?
yann2 5y ago
Correct. Rationality is Bounded. That fact won a Nobel Prize - https://en.wikipedia.org/wiki/Herbert_A._Simon https://en.wikipedia.org/wiki/Herbert_A._Simon The recommendation of the theory is if you cant be rational about a specific problem pick another problem, preferably a simpler problem. Unfortunately lots of chimps in the troupe are incapable of doing that and therefore we shall always have drama.
- WastingMyTime89 5y ago
  > Correct. Rationality is Bounded. That fact won a Nobel Prize. It's a model not a fact. As a model, it can't really be correct only more or less accurate.
  - abc_lisper 5y ago
    any model better than naive intuition is better imo
  - loopz 5y ago
    It's a fact many people believe themselves rational and use models expecting rational actors. Proof lacks that it actually works that way. The opposite is often most probable since you rarely have perfect knowledge in practice. Exceptions can be games like tic-tac-toe and chess.
    - WastingMyTime89 5y ago
      > Proof lacks that it actually works that way I mean everyone know it doesn't really work that way. The actual question is: does viewing the average actor as trying to perfectly optimise their utility function using all the information available constitute a good estimation of how actors work in aggregate and does it yield accurate and interesting predictions? The real insight of Simon in Models of Man is not that actors are not in fact perfectly rational. It's that you can actually model the limits of actors while keeping a fairly rigorous and manageable formalization.
      - loopz 5y ago
        Sure, but such models are hackable / breakable, just by breaking the rules or reinventing the game.
      - techbio 5y ago
        I'll have to read Simon, but I'll be looking for some stable types of responses to uncertainty, not rational but adaptive: reversion to past/training, confusion, overreaction, etc. ie something prevalent enough that it can be reasoned about.
  - zepto 5y ago
    > It's a model not a fact. As a model, it can't really be correct only more or less accurate. This is not true. Models of an external world may be only more or less accurate, but models of other models may be true or false. Mathematical proofs rely on this. Rationality itself is a model so models of rationality may be true or false.
- vendiddy 5y ago
  Could someone explain in laymen terms what "bounded" rationality means?
  - glial 5y ago
    Rationality can be re-phrased as coming up with the optimal solution to a problem. If you only have finite compute/memory/time, your solution is 'bounded' by those constraints - i.e. your job is now find the best solution possible given the constraints.
- btilly 5y ago
  That Nobel was won in 1978, and is based on the fact that in practice we can't be rational. The NP demonstrations that, in theory, updating a Bayesian is a computationally infeasible problem was G. F. Cooper in 1990 (for Bayesian Networks). The stronger result that approximating the update is also computationally infeasible was Dagum, P. & Luby, M., 1993. So Simon's work relates to what I said, but isn't based on it.
- ggm 5y ago
  I've made it a life rule to try to avoid conversations with people who use "correct" to respond to statements, unless they are in the role of a teacher. Tell me, are you aware of the myriad of alternate words to express your agreement with somebody else aside from correct? You aren't here as judge of the right or wrong. Semantically, philosophically, you're expressing agreement not correctness. Or .. am I incorrect...?
- carrolldunham 5y ago
  * A Nobel Memorial Prize in Economic Sciences, officially the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel, commonly but falsely referred to as the Nobel Prize in Economics
lalaithion 5y ago
That's why it's called "less wrong". The goal isn't to be perfect, the goal is to do better. To be less wrong.
- wombatmobile 5y ago
  > the goal is to do better Why is that "the" goal? Who sets "the" goal?
  - voxic11 5y ago
    Its the goal of the LessWrong/Rationalist community.
    - analog31 5y ago
      Less Wrong seems to be a manifestation of a thing that comes in cycles: Something triggers the rise of a "rationalist" movement, including possibly a new evangelist or a new medium. Eventually, rationalism and rational people end up at a standoff. Then the whole thing repeats itself after a period of time. I'm probably rational enough but also can't make sense of much of the rationalist literature, so I simply follow my own compass and hope for the best. I'm skeptical of Bayes Theater.
  - Nav_Panel 5y ago
    Yudkowsky and other prominent contributors set "the" goal as a sort of revealed wisdom regarding their speculations about what a super powerful AI will do. Pragmatically, the goals themselves appeal to individuals who want to maintain conventional (liberal) morality yet also position themselves as superior, typically as a form of compensation.
    - nitrogen 5y ago
      position themselves as superior This is why we can't have nice things. Any time someone tries to find more effective ways of making good decisions or accomplishing their goals, someone has to bring out the most tortured cynical interpretation to tear them down.
polote 5y ago
Well, it depends on the topic. Choosing rationally between the most green of two tomatoes is easy. You are limited by your ability to distinguish colors but you can still decide rationally. Not all questions have answers, if you want to be rational when you are asked to answer those question, you can just say "I dont know" At the beginning of the pandemic, when politics were saying mask dont work. You could just say, well, if we transmit covid by air, then putting something in front of my mouth is going to decrease the spread. That's what is being rational. Of course that's not going to be all the time the good answer, but you have still thought rationally. I'm not really sure what you are trying to prove. Of curse being rational is possible. All people are rational for most of their decisions.
- not2b 5y ago
  A rationalist would recognize that we update our beliefs as new evidence is available and not attack people for having erroneous beliefs before that evidence was available. The "masks don't work" advice was active for a short time in March 2020 and almost immediately dumped. They thought at the time that only n95 masks would be good enough, these masks were in short supply and health care workers needed them, this was the "politics" of it. But by mid March 2020 people were already being encouraged to make cloth masks and how to do it. That is when my daughter got out the sewing machine and made a bunch, based on instructions from nurses.
  - polote 5y ago
    There was no new evidence. I'm not sure we have even learned anything regarding the efficacy of masks trough the pandemic. All what we know was already known prior of it.
    - kbelder 5y ago
      'We' as the scientific community may not have, but 'we' the unwashed public learned much.
      - polote 5y ago
snarf21 5y ago
Agreed. The world is too complicated. There is too much noise. It might be possible to become fairly knowledgeable about a single issue but it would need to amount to an obsession. This is why we are so keen to belong. We've always tried to apply the "wisdom of the crowds". It is why people latch onto one viewpoint or the other, e.g. Red/Blue, FOX/CNN, etc., it takes all the work out of it. Once you find a source that you agree with on even ONE issue, just blindly trust/agree with them for everything. We'd rather spend our time streaming shows and living life than investing into deep knowledge of any subject. Take a non political example: How safe are whole tomatoes to eat? What did the grocery store spray on them? Is it safe? Will it wash off? What about the warehouse were they were stored for months, what did they put on them to keep them from spoiling? What about the farmer, what did they spray on them to protect against pests? What is in the water, is it safe? Now we're ready to eat: Does anyone in my family have any kind of intolerance to raw tomatoes? And this is a pretty simple toy example.... In general, we've collectively decided to trust in the good in people. We hope that if something is bad/lie/harmful, then someone in the know will raise the alarm for the group.
lisper 5y ago
Ironically, some of the most irrational people I know are the ones who profess to hew to rationality, to the point where, in certain circles, "rationality" has become a sort of cult. This is particularly evident in militant anti-theism (whose adherents insist that the only possible explanation for someone believing in God is that they are idiots or otherwise mentally deficient), hard-core libertarians (who, ironically, end up politically aligned with hard-core fundamentalist Christians, at least in the U.S.) and a particularly weird strain of this disease that causes people to subscribe to (and actively proselytize!) the many-worlds interpretation of quantum mechanics. It's bizarre, and unendingly frustrating. Sometimes I feel like I'm the only rational creature in the universe because, of course, none of my beliefs are anything at all like theirs.
- btilly 5y ago
  How do you know that they are the ones who are being irrational here, and not you? This is a serious question. We should always challenge our preconceptions. To take your examples: 1. Traditional Judeo-Christian religions all claim we should believe because of claims made in holy books of questionable provenance, held by primitive people who believed things like (for example) disease being caused by demons. What rational reason is there for believing these holy books to be particularly truthful? (I was careful to not include Buddhism, whose basis is in experiences that people have while in altered states of consciousness from meditation.) 2. The shortcomings of libertarianism involve various tragedies of the commons. (My favorite book on this being, The Logic of Collective Action.) However the evidence in favor of most government interventions is rather weak. And the evidence is very strong that well-intended government interventions predictably will, after regulatory capture, wind creating severe problems of their own. How do you know that the interventions which you like will actually lead to good results? (Note, both major US parties are uneasy coalitions of convenience kept together through the only electoral realities of winner takes all. On the left, big labor and environmentalism are also uncomfortable bedfellows.) 3. To the extent that the observer is described by quantum mechanics, many-worlds is provably a correct description of the process of observation. In the absence of concrete evidence that quantum mechanics breaks down for observers like us, what rational reason is there to advocate for any other interpretation? (The fact that it completely violates our preconceptions about how the world should work is an emotional argument, not a rational one.)
6gvONxR4sf7o 5y ago
That’s true for arbitrary graphs, but I don’t believe it’s practically relevant here any more than the fact that I can’t compute the first 10000 digits of pi in my head is. We are much worse than our computational limits.
varjag 5y ago
Thing is the common failures of rational thinking are not approaching any computational limits. Witness the dumbassery of the past two years.
- irrational 5y ago
  Past 5-6 years you mean.
  - jaredhansen 5y ago
    All past years you mean. It's not exactly a recent phenomenon.
    - mcguire 5y ago
      Life would be easier if we could agree on one rational decisions in history and then just repeat it as necessary.
    - irrational 5y ago
      It's both sides, right? Right....
      - mistermann 5y ago
        Logically, "both" sides seems like the correct answer to me.
deleted 5y ago
[deleted]
Jensson 5y ago
Being rational includes being rational about computation power and heuristics used on a specific choice. Therefore irrational is when people make completely stupid choices that aren't computationally hard to make, not that people can't solve NP-hard problems.
ulucs 5y ago
Why bother reasoning with NP-hardness when you can just invoke incompleteness? No brain power limitations are needed.
- drdeca 5y ago
  Because incompleteness isn’t really relevant here? Are you just saying “people aren’t logically omniscient, and can’t be because of incompleteness”?
Tenoke 5y ago
Nobody is disputing this. You can, however, clearly be more or less 'rational', adopt better or worse heuristics, etc. which is what you attempt to get from reading LessWrong or Kahneman.
- nicoburns 5y ago
  But what constitutes a better heuristic is context dependent. In particular, if I must make a decision in a time-constrained manner then any heuristic that blows the time budget is going to be worse even if it would be better given more time. And one can't really know in advance how much time spent thinking is optimal. So one has to pick a strategy. The fact that humans have evolved to use a variety of strategies fast and slow (depending on the human) suggests that there is no single optimal strategy. See also Gigerenzer's Ecological Rationality.
  - Tenoke 5y ago
    Sure, but I doubt you actually think that everyone is already operating as well as they can within the contexts they are placed. There's definitely room for improvement. There are all sorts of scenarios where even knowing nearly optimal techniques outcomes can be improved by going the TimSort way due to the context you most often find yourself in.
joe_the_user 5y ago
Not mention we don't actually know with certain the probability of even simple things. The average person is almost never reasoning about simple, repeatable sequences of events. You know there's a chance of your car breaking down each day but you don't know the probability of that event and yet you still deal with that possibility.
nonameiguess 5y ago
We should note the limitations of Bayes as well. I already responded to another comment in this thread that giving an update procedure based on seeing new evidence is necessarily bounded in how quickly it can get you to beliefs more likely to be true by your ability to actually gather that evidence or possibly even to generate it if it doesn't already exist. We don't have any perfect algorithms for doing that, and it is of course not a purely computational problem anyway. Take general relativity. It was proposed in 1915 and only confirmed in the very strong gravitational field limit in 2016, because that was our first opportunity to observe a black hole merger, which is not something we have the ability to recreate in a lab. Even beyond the hard process bottleneck on creating or lucking upon events that produce the evidence we need, however, there is also the limitation that Bayes only gives you a probability. It doesn't give you a decision theory or even a thresholding function. For those, you need a whole lot of other things like utility functions, discount rates, receiver operating characteristics and an understanding of asymmetric costs of false positives versus false negatives, that are often different for each decision domain. And, of course, to get a utility function meaningful for humans, you need values. There is no algorithm that can give you values. They're just there as a basic primitive input to all other decision making procedures, yet they often conflict in ways that cannot be reconciled even within a single person, let alone across a society of many people.
strulovich 5y ago
NP hard problems get abused for justifying things they cannot. An NP hard problem, even if it cannot be approximated does not mean the average input cannot be solved efficiently. Examples: - An NP hard problem is not sufficient for building crypto. - Type solving for many programming languages is EXP TIME complete, yet those languages prosper and compile just fine. Beware the idea of taking a mathematical concept and proof and inducing from it to the world outside the model.
- DiggyJohnson 5y ago
  First of all I do see that you called it an example; I don't think you're straw-manning or anything: I think using chaos theory / Bayesian concepts is a significantly better metaphor for "life as we experience it" than it is for the examples you gave.
- nostrademons 5y ago
  And human beings make approximate solutions for the average input all the time. That's what gut feelings, instincts, heuristics, and motivated reasoning are, along with all the other shortcuts we take to function in daily life. The article is asking why it's so hard to be rational though, i.e. follow a logically-valid set of inferences forward to an unambiguous conclusion. Assuming one of your premises is that correct rationality implies reasoning statistically about a network of interrelated beliefs, the uncomputability of a Bayesian net is relevant to that.
  - munk-a 5y ago
    Following a fully logically valid set of inferences seems extremely inefficient - we need to make decisions constantly and relying on short hand for most of those seems perfectly rational - it's rational to trust irrational gut feelings for most unimportant decisions because trying to fully prove all actions is a fool's errand. I think the article is more focused on those big decisions where rationality is certainly warranted and so often ignored. People who are highly skilled at life have developed their gut feelings and instincts to be able to determine which decisions they really need to sit down and think hard about and which ones they can mostly ignore. When most people buy their first house the decision is so immensely large and represents such a high value (more than half a million at least for a lot of city folk) that there is a desire to detach from it to free yourself from responsibility - since you cannot sanely account for all factors it is "safer" to protect your ego by delegating the decision entirely on your id - doing so allows you to, post de facto, entirely free yourself from any responsibility of your poor decision. This, I think, is the main factor we need to fight against to make rational decisions - you must accept failure and be willing to be wrong without shame. Do your best to evaluate your options on important decisions and realize that there are a number of decisions you obviously can't fully rationalize out - you can only make your best attempt. But realize that making your best attempt and being wrong - as much as it might hurt your ego - is a better alternative than "letting it ride" and being able to stand blameless on the far end. The fight for rationality is mostly a fight against emotional fragility and intellectual laziness.
tisthetruth 5y ago
Not being jacked up on sugar and caffeine can help tremendously. I would still like to see some studies which delve into whether sugar and caffeine are catalysts for biasing us towards system 1 and how they affect system 2, mindfulness, patience, etc...
kazinator 5y ago
> I maintain that it isn't just hard, it is computationally impossible. I further maintain that it's definitionally impossible. Before we find it computationally impossible, we will find that we can't write the a complete, detailed requirements specification defining what rational is. (Of course, we recognize egregious irrationality when we see it; that's not what I mean; you can't just define rationality as the opposite of that.) People can behave rationally (or not) with respect to some stated values that they have. But those can be arbitrary. So the requirement specification for rationality has to refer to a "configuration space", so to speak, where we program these values. This means that the output is dependent on it; we can't write some absolute test case for rationality that doesn't include this. Problem is, people with different values look at each other's values and point to them and say, "those values are irrational; those people should adopt my values instead".
- UnFleshedOne 5y ago
  You can't say values are irrational -- they just are. If you really like paperclips, no amount of logic can tell you otherwise. What logic can tell you (and other people could), is that your values conflict with each other and you have to balance one against another. Turning whole universe into paperclips is counterproductive if you also value pins. If you literally have no value the other person is basing their arguments on, then they can't convince you to have it. Luckily we get our values from bunch of heuristics developed through millions of years of biological and social evolution, so we mostly have the same ones, just with different relative weights. Won't be true if we ever meet (or make) some other sentient critters.
  - kazinator 5y ago
    > You can't say values are irrational People basically do say that, though. (Values can be contradictory/inconsistent. E.g. you say you value self-preservation, but you also enjoy whacking your head with a hammer. That would be a kind of irrational. That's not what I'm referring to though.)
    - UnFleshedOne 5y ago
didibus 5y ago
That might be true given a single brain, but we as a species have access to billions of brains. The question is, can we organize and educate ourselves so we can leverage that parallel power and let each person become experts in their areas with proper trusts and incentives? And manage to pass along the previous generation computation to the next, without corrupting the data? Edit: And I forgot all the tools we've designed to help us compute all that, of which I'd count math as a tool to help us compute, and computers as another.
jdmichal 5y ago
In addition to Thinking, Fast and Slow, I'd recommend Annie Duke's Thinking in Bets. It builds on literature such as Thinking, Fast and Slow to discuss the separation of decisions and results. Specifically, thanks to luck, the quality of the decision is not always represented by the quality of the result. And in order to learn, one has to be able to recognize good and bad decisions regardless of results. This seems to be a pretty good overview: https://www.athenarium.com/thinking-in-bets-annie-duke/ https://www.athenarium.com/thinking-in-bets-annie-duke/
heresie-dabord 5y ago
> I maintain that it isn't just hard, it is computationally impossible. [...] The effort of doing better is still worthwhile. But the goal itself is unachievable. The goal of rational thinking is not some conceit of perfection [1] but debugging the runtime for a better result. Humans are in fact very good at communication and at debugging language errors. They have evolved a rational capacity. It can evidently be developed but it needs to be exercised. This is where hypothesis of an educational system often enters the discussion. [1] Galef and others call the "Star Trek" Spock character a Vulcan Strawman or Straw Vulcan. https://en.wikipedia.org/wiki/Julia_Galef https://en.wikipedia.org/wiki/Julia_Galef
threatofrain 5y ago
Daniel Kahneman has soured on his own System 1/2 theory, plus his original theory discussed bounded rationality and not the kind of objective rationality which fell out of favor in econ literature a long time ago.
- ad8e 5y ago
  I'm interested in learning more. Is there a resource where I can see Kahneman's changed thoughts?
garbagetime 5y ago
> We should all know that given a belief about the world, and evidence, Bayes' Theorem describes how to update our beliefs. Should we? What of the problem of induction?
- btilly 5y ago
  The problem of induction is a problem of how we may create a prior out of raw experience. And neither logic nor mathematics offers a solution. In practice, however, we do. But, as any parent should know, we don't do it through a rational process.
AndrewKemendo 5y ago
This nails it. I'd go further to say that there are real world issues that compound the variables. Namely that individual actions increasingly have global consequences eg. individual purchasing behaviors have externalities that the market is not pricing in and thus fall to the consumer to have to calculate. Further, given that these global issues these kinds of calculations are game theoretic by their nature, making it even more complicated.
yibg 5y ago
Being NP-hard doesn't make it computationally impossible in all cases though. So while it might be computationally impossible to be rational in ALL cases, it could be computationally possible to be rational in some (or even many) cases. I think that's the goal to strive for.
dwd 5y ago
Memory and learning is additive - we don't have a delete key, except for where a model can be completely replaced with something new, which is usually at that simple fact level - but it's then assimilated into the rest of what we believe (like a wave function collapse) but it allows for discordant ideas at a distance - irrationality!

by btilly 5y ago

kirso 5y ago

Does this mean then that all the mental models are basically useless because the outcome may be irrational?

yann2 5y ago

Correct. Rationality is Bounded. That fact won a Nobel Prize - https://en.wikipedia.org/wiki/Herbert_A._Simon https://en.wikipedia.org/wiki/Herbert_A._Simon The recommendation of the theory is if you cant be rational about a specific problem pick another problem, preferably a simpler problem. Unfortunately lots of chimps in the troupe are incapable of doing that and therefore we shall always have drama.

WastingMyTime89 5y ago
> Correct. Rationality is Bounded. That fact won a Nobel Prize. It's a model not a fact. As a model, it can't really be correct only more or less accurate.
- abc_lisper 5y ago
  any model better than naive intuition is better imo
- loopz 5y ago
  It's a fact many people believe themselves rational and use models expecting rational actors. Proof lacks that it actually works that way. The opposite is often most probable since you rarely have perfect knowledge in practice. Exceptions can be games like tic-tac-toe and chess.
  - WastingMyTime89 5y ago
    > Proof lacks that it actually works that way I mean everyone know it doesn't really work that way. The actual question is: does viewing the average actor as trying to perfectly optimise their utility function using all the information available constitute a good estimation of how actors work in aggregate and does it yield accurate and interesting predictions? The real insight of Simon in Models of Man is not that actors are not in fact perfectly rational. It's that you can actually model the limits of actors while keeping a fairly rigorous and manageable formalization.
    - loopz 5y ago
      Sure, but such models are hackable / breakable, just by breaking the rules or reinventing the game.
    - techbio 5y ago
      I'll have to read Simon, but I'll be looking for some stable types of responses to uncertainty, not rational but adaptive: reversion to past/training, confusion, overreaction, etc. ie something prevalent enough that it can be reasoned about.
- zepto 5y ago
  > It's a model not a fact. As a model, it can't really be correct only more or less accurate. This is not true. Models of an external world may be only more or less accurate, but models of other models may be true or false. Mathematical proofs rely on this. Rationality itself is a model so models of rationality may be true or false.
vendiddy 5y ago
Could someone explain in laymen terms what "bounded" rationality means?
- glial 5y ago
  Rationality can be re-phrased as coming up with the optimal solution to a problem. If you only have finite compute/memory/time, your solution is 'bounded' by those constraints - i.e. your job is now find the best solution possible given the constraints.
btilly 5y ago
That Nobel was won in 1978, and is based on the fact that in practice we can't be rational. The NP demonstrations that, in theory, updating a Bayesian is a computationally infeasible problem was G. F. Cooper in 1990 (for Bayesian Networks). The stronger result that approximating the update is also computationally infeasible was Dagum, P. & Luby, M., 1993. So Simon's work relates to what I said, but isn't based on it.
ggm 5y ago
I've made it a life rule to try to avoid conversations with people who use "correct" to respond to statements, unless they are in the role of a teacher. Tell me, are you aware of the myriad of alternate words to express your agreement with somebody else aside from correct? You aren't here as judge of the right or wrong. Semantically, philosophically, you're expressing agreement not correctness. Or .. am I incorrect...?
carrolldunham 5y ago
* A Nobel Memorial Prize in Economic Sciences, officially the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel, commonly but falsely referred to as the Nobel Prize in Economics

lalaithion 5y ago

That's why it's called "less wrong". The goal isn't to be perfect, the goal is to do better. To be less wrong.

wombatmobile 5y ago
> the goal is to do better Why is that "the" goal? Who sets "the" goal?
- voxic11 5y ago
  Its the goal of the LessWrong/Rationalist community.
  - analog31 5y ago
    Less Wrong seems to be a manifestation of a thing that comes in cycles: Something triggers the rise of a "rationalist" movement, including possibly a new evangelist or a new medium. Eventually, rationalism and rational people end up at a standoff. Then the whole thing repeats itself after a period of time. I'm probably rational enough but also can't make sense of much of the rationalist literature, so I simply follow my own compass and hope for the best. I'm skeptical of Bayes Theater.
- Nav_Panel 5y ago
  Yudkowsky and other prominent contributors set "the" goal as a sort of revealed wisdom regarding their speculations about what a super powerful AI will do. Pragmatically, the goals themselves appeal to individuals who want to maintain conventional (liberal) morality yet also position themselves as superior, typically as a form of compensation.
  - nitrogen 5y ago
    position themselves as superior This is why we can't have nice things. Any time someone tries to find more effective ways of making good decisions or accomplishing their goals, someone has to bring out the most tortured cynical interpretation to tear them down.

polote 5y ago

Well, it depends on the topic. Choosing rationally between the most green of two tomatoes is easy. You are limited by your ability to distinguish colors but you can still decide rationally. Not all questions have answers, if you want to be rational when you are asked to answer those question, you can just say "I dont know" At the beginning of the pandemic, when politics were saying mask dont work. You could just say, well, if we transmit covid by air, then putting something in front of my mouth is going to decrease the spread. That's what is being rational. Of course that's not going to be all the time the good answer, but you have still thought rationally. I'm not really sure what you are trying to prove. Of curse being rational is possible. All people are rational for most of their decisions.

not2b 5y ago
A rationalist would recognize that we update our beliefs as new evidence is available and not attack people for having erroneous beliefs before that evidence was available. The "masks don't work" advice was active for a short time in March 2020 and almost immediately dumped. They thought at the time that only n95 masks would be good enough, these masks were in short supply and health care workers needed them, this was the "politics" of it. But by mid March 2020 people were already being encouraged to make cloth masks and how to do it. That is when my daughter got out the sewing machine and made a bunch, based on instructions from nurses.
- polote 5y ago
  There was no new evidence. I'm not sure we have even learned anything regarding the efficacy of masks trough the pandemic. All what we know was already known prior of it.
  - kbelder 5y ago
    'We' as the scientific community may not have, but 'we' the unwashed public learned much.
    - polote 5y ago

snarf21 5y ago

Agreed. The world is too complicated. There is too much noise. It might be possible to become fairly knowledgeable about a single issue but it would need to amount to an obsession. This is why we are so keen to belong. We've always tried to apply the "wisdom of the crowds". It is why people latch onto one viewpoint or the other, e.g. Red/Blue, FOX/CNN, etc., it takes all the work out of it. Once you find a source that you agree with on even ONE issue, just blindly trust/agree with them for everything. We'd rather spend our time streaming shows and living life than investing into deep knowledge of any subject. Take a non political example: How safe are whole tomatoes to eat? What did the grocery store spray on them? Is it safe? Will it wash off? What about the warehouse were they were stored for months, what did they put on them to keep them from spoiling? What about the farmer, what did they spray on them to protect against pests? What is in the water, is it safe? Now we're ready to eat: Does anyone in my family have any kind of intolerance to raw tomatoes? And this is a pretty simple toy example.... In general, we've collectively decided to trust in the good in people. We hope that if something is bad/lie/harmful, then someone in the know will raise the alarm for the group.

lisper 5y ago

Ironically, some of the most irrational people I know are the ones who profess to hew to rationality, to the point where, in certain circles, "rationality" has become a sort of cult. This is particularly evident in militant anti-theism (whose adherents insist that the only possible explanation for someone believing in God is that they are idiots or otherwise mentally deficient), hard-core libertarians (who, ironically, end up politically aligned with hard-core fundamentalist Christians, at least in the U.S.) and a particularly weird strain of this disease that causes people to subscribe to (and actively proselytize!) the many-worlds interpretation of quantum mechanics. It's bizarre, and unendingly frustrating. Sometimes I feel like I'm the only rational creature in the universe because, of course, none of my beliefs are anything at all like theirs.

btilly 5y ago
How do you know that they are the ones who are being irrational here, and not you? This is a serious question. We should always challenge our preconceptions. To take your examples: 1. Traditional Judeo-Christian religions all claim we should believe because of claims made in holy books of questionable provenance, held by primitive people who believed things like (for example) disease being caused by demons. What rational reason is there for believing these holy books to be particularly truthful? (I was careful to not include Buddhism, whose basis is in experiences that people have while in altered states of consciousness from meditation.) 2. The shortcomings of libertarianism involve various tragedies of the commons. (My favorite book on this being, The Logic of Collective Action.) However the evidence in favor of most government interventions is rather weak. And the evidence is very strong that well-intended government interventions predictably will, after regulatory capture, wind creating severe problems of their own. How do you know that the interventions which you like will actually lead to good results? (Note, both major US parties are uneasy coalitions of convenience kept together through the only electoral realities of winner takes all. On the left, big labor and environmentalism are also uncomfortable bedfellows.) 3. To the extent that the observer is described by quantum mechanics, many-worlds is provably a correct description of the process of observation. In the absence of concrete evidence that quantum mechanics breaks down for observers like us, what rational reason is there to advocate for any other interpretation? (The fact that it completely violates our preconceptions about how the world should work is an emotional argument, not a rational one.)

6gvONxR4sf7o 5y ago

That’s true for arbitrary graphs, but I don’t believe it’s practically relevant here any more than the fact that I can’t compute the first 10000 digits of pi in my head is. We are much worse than our computational limits.

varjag 5y ago

Thing is the common failures of rational thinking are not approaching any computational limits. Witness the dumbassery of the past two years.

irrational 5y ago
Past 5-6 years you mean.
- jaredhansen 5y ago
  All past years you mean. It's not exactly a recent phenomenon.
  - mcguire 5y ago
    Life would be easier if we could agree on one rational decisions in history and then just repeat it as necessary.
  - irrational 5y ago
    It's both sides, right? Right....
    - mistermann 5y ago
      Logically, "both" sides seems like the correct answer to me.

deleted 5y ago

[deleted]

Jensson 5y ago

Being rational includes being rational about computation power and heuristics used on a specific choice. Therefore irrational is when people make completely stupid choices that aren't computationally hard to make, not that people can't solve NP-hard problems.

ulucs 5y ago

Why bother reasoning with NP-hardness when you can just invoke incompleteness? No brain power limitations are needed.

drdeca 5y ago
Because incompleteness isn’t really relevant here? Are you just saying “people aren’t logically omniscient, and can’t be because of incompleteness”?

Tenoke 5y ago

Nobody is disputing this. You can, however, clearly be more or less 'rational', adopt better or worse heuristics, etc. which is what you attempt to get from reading LessWrong or Kahneman.

nicoburns 5y ago
But what constitutes a better heuristic is context dependent. In particular, if I must make a decision in a time-constrained manner then any heuristic that blows the time budget is going to be worse even if it would be better given more time. And one can't really know in advance how much time spent thinking is optimal. So one has to pick a strategy. The fact that humans have evolved to use a variety of strategies fast and slow (depending on the human) suggests that there is no single optimal strategy. See also Gigerenzer's Ecological Rationality.
- Tenoke 5y ago
  Sure, but I doubt you actually think that everyone is already operating as well as they can within the contexts they are placed. There's definitely room for improvement. There are all sorts of scenarios where even knowing nearly optimal techniques outcomes can be improved by going the TimSort way due to the context you most often find yourself in.

joe_the_user 5y ago

Not mention we don't actually know with certain the probability of even simple things. The average person is almost never reasoning about simple, repeatable sequences of events. You know there's a chance of your car breaking down each day but you don't know the probability of that event and yet you still deal with that possibility.

nonameiguess 5y ago

We should note the limitations of Bayes as well. I already responded to another comment in this thread that giving an update procedure based on seeing new evidence is necessarily bounded in how quickly it can get you to beliefs more likely to be true by your ability to actually gather that evidence or possibly even to generate it if it doesn't already exist. We don't have any perfect algorithms for doing that, and it is of course not a purely computational problem anyway. Take general relativity. It was proposed in 1915 and only confirmed in the very strong gravitational field limit in 2016, because that was our first opportunity to observe a black hole merger, which is not something we have the ability to recreate in a lab. Even beyond the hard process bottleneck on creating or lucking upon events that produce the evidence we need, however, there is also the limitation that Bayes only gives you a probability. It doesn't give you a decision theory or even a thresholding function. For those, you need a whole lot of other things like utility functions, discount rates, receiver operating characteristics and an understanding of asymmetric costs of false positives versus false negatives, that are often different for each decision domain. And, of course, to get a utility function meaningful for humans, you need values. There is no algorithm that can give you values. They're just there as a basic primitive input to all other decision making procedures, yet they often conflict in ways that cannot be reconciled even within a single person, let alone across a society of many people.

strulovich 5y ago

NP hard problems get abused for justifying things they cannot. An NP hard problem, even if it cannot be approximated does not mean the average input cannot be solved efficiently. Examples: - An NP hard problem is not sufficient for building crypto. - Type solving for many programming languages is EXP TIME complete, yet those languages prosper and compile just fine. Beware the idea of taking a mathematical concept and proof and inducing from it to the world outside the model.

DiggyJohnson 5y ago
First of all I do see that you called it an example; I don't think you're straw-manning or anything: I think using chaos theory / Bayesian concepts is a significantly better metaphor for "life as we experience it" than it is for the examples you gave.
nostrademons 5y ago
And human beings make approximate solutions for the average input all the time. That's what gut feelings, instincts, heuristics, and motivated reasoning are, along with all the other shortcuts we take to function in daily life. The article is asking why it's so hard to be rational though, i.e. follow a logically-valid set of inferences forward to an unambiguous conclusion. Assuming one of your premises is that correct rationality implies reasoning statistically about a network of interrelated beliefs, the uncomputability of a Bayesian net is relevant to that.
- munk-a 5y ago
  Following a fully logically valid set of inferences seems extremely inefficient - we need to make decisions constantly and relying on short hand for most of those seems perfectly rational - it's rational to trust irrational gut feelings for most unimportant decisions because trying to fully prove all actions is a fool's errand. I think the article is more focused on those big decisions where rationality is certainly warranted and so often ignored. People who are highly skilled at life have developed their gut feelings and instincts to be able to determine which decisions they really need to sit down and think hard about and which ones they can mostly ignore. When most people buy their first house the decision is so immensely large and represents such a high value (more than half a million at least for a lot of city folk) that there is a desire to detach from it to free yourself from responsibility - since you cannot sanely account for all factors it is "safer" to protect your ego by delegating the decision entirely on your id - doing so allows you to, post de facto, entirely free yourself from any responsibility of your poor decision. This, I think, is the main factor we need to fight against to make rational decisions - you must accept failure and be willing to be wrong without shame. Do your best to evaluate your options on important decisions and realize that there are a number of decisions you obviously can't fully rationalize out - you can only make your best attempt. But realize that making your best attempt and being wrong - as much as it might hurt your ego - is a better alternative than "letting it ride" and being able to stand blameless on the far end. The fight for rationality is mostly a fight against emotional fragility and intellectual laziness.

tisthetruth 5y ago

Not being jacked up on sugar and caffeine can help tremendously. I would still like to see some studies which delve into whether sugar and caffeine are catalysts for biasing us towards system 1 and how they affect system 2, mindfulness, patience, etc...

kazinator 5y ago

> I maintain that it isn't just hard, it is computationally impossible. I further maintain that it's definitionally impossible. Before we find it computationally impossible, we will find that we can't write the a complete, detailed requirements specification defining what rational is. (Of course, we recognize egregious irrationality when we see it; that's not what I mean; you can't just define rationality as the opposite of that.) People can behave rationally (or not) with respect to some stated values that they have. But those can be arbitrary. So the requirement specification for rationality has to refer to a "configuration space", so to speak, where we program these values. This means that the output is dependent on it; we can't write some absolute test case for rationality that doesn't include this. Problem is, people with different values look at each other's values and point to them and say, "those values are irrational; those people should adopt my values instead".

UnFleshedOne 5y ago
You can't say values are irrational -- they just are. If you really like paperclips, no amount of logic can tell you otherwise. What logic can tell you (and other people could), is that your values conflict with each other and you have to balance one against another. Turning whole universe into paperclips is counterproductive if you also value pins. If you literally have no value the other person is basing their arguments on, then they can't convince you to have it. Luckily we get our values from bunch of heuristics developed through millions of years of biological and social evolution, so we mostly have the same ones, just with different relative weights. Won't be true if we ever meet (or make) some other sentient critters.
- kazinator 5y ago
  > You can't say values are irrational People basically do say that, though. (Values can be contradictory/inconsistent. E.g. you say you value self-preservation, but you also enjoy whacking your head with a hammer. That would be a kind of irrational. That's not what I'm referring to though.)
  - UnFleshedOne 5y ago

didibus 5y ago

That might be true given a single brain, but we as a species have access to billions of brains. The question is, can we organize and educate ourselves so we can leverage that parallel power and let each person become experts in their areas with proper trusts and incentives? And manage to pass along the previous generation computation to the next, without corrupting the data? Edit: And I forgot all the tools we've designed to help us compute all that, of which I'd count math as a tool to help us compute, and computers as another.

jdmichal 5y ago

In addition to Thinking, Fast and Slow, I'd recommend Annie Duke's Thinking in Bets. It builds on literature such as Thinking, Fast and Slow to discuss the separation of decisions and results. Specifically, thanks to luck, the quality of the decision is not always represented by the quality of the result. And in order to learn, one has to be able to recognize good and bad decisions regardless of results. This seems to be a pretty good overview: https://www.athenarium.com/thinking-in-bets-annie-duke/ https://www.athenarium.com/thinking-in-bets-annie-duke/

heresie-dabord 5y ago

> I maintain that it isn't just hard, it is computationally impossible. [...] The effort of doing better is still worthwhile. But the goal itself is unachievable. The goal of rational thinking is not some conceit of perfection [1] but debugging the runtime for a better result. Humans are in fact very good at communication and at debugging language errors. They have evolved a rational capacity. It can evidently be developed but it needs to be exercised. This is where hypothesis of an educational system often enters the discussion. [1] Galef and others call the "Star Trek" Spock character a Vulcan Strawman or Straw Vulcan. https://en.wikipedia.org/wiki/Julia_Galef https://en.wikipedia.org/wiki/Julia_Galef

threatofrain 5y ago

Daniel Kahneman has soured on his own System 1/2 theory, plus his original theory discussed bounded rationality and not the kind of objective rationality which fell out of favor in econ literature a long time ago.

ad8e 5y ago
I'm interested in learning more. Is there a resource where I can see Kahneman's changed thoughts?

garbagetime 5y ago

> We should all know that given a belief about the world, and evidence, Bayes' Theorem describes how to update our beliefs. Should we? What of the problem of induction?

btilly 5y ago
The problem of induction is a problem of how we may create a prior out of raw experience. And neither logic nor mathematics offers a solution. In practice, however, we do. But, as any parent should know, we don't do it through a rational process.

AndrewKemendo 5y ago

This nails it. I'd go further to say that there are real world issues that compound the variables. Namely that individual actions increasingly have global consequences eg. individual purchasing behaviors have externalities that the market is not pricing in and thus fall to the consumer to have to calculate. Further, given that these global issues these kinds of calculations are game theoretic by their nature, making it even more complicated.

yibg 5y ago

Being NP-hard doesn't make it computationally impossible in all cases though. So while it might be computationally impossible to be rational in ALL cases, it could be computationally possible to be rational in some (or even many) cases. I think that's the goal to strive for.

dwd 5y ago

Memory and learning is additive - we don't have a delete key, except for where a model can be completely replaced with something new, which is usually at that simple fact level - but it's then assimilated into the rest of what we believe (like a wave function collapse) but it allows for discordant ideas at a distance - irrationality!