4 ms·

In AI this is a little bit known as the rationality paradox. People generally agree that the following formula is the definition of acting rationally : argmax(

by waps 13y ago

In AI this is a little bit known as the rationality paradox. People generally agree that the following formula is the definition of acting rationally : argmax(sum(probability_of_event * utility_of_event) given action) So do you open a door ? There could be a bear behind the door that will eat you if you open the door. The probability of that is not zero (and even if it was, it would not matter). The utility of that is -inf. The utility of not opening the door is also -inf. There could be a bear behind you and the door could be your only way to safety. So a rational actor would never do anything at all, as the rational function would never converge. Ergo, nobody and nothing is rational (as soon as the world has a certain minimal level of complexity). There are various suggestions given out of this conundrum, but none are even mildly satisfying. For example, one could claim that you don't know those odds, so you don't care. But that is the same as saying you can't really be rational, so it's not a solution.

FigBug 13y ago
In "The Man Who mistook His Wife for a Hat" (http://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_Wife_for_a_Hat http://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_Wife_fo...), a book on mental disorders, there is a case of a man who can only make rational decisions. He is unable to function, he can't get past what to wear or what to eat in the morning. Even this simplest decisions take him hours.
- taejo 13y ago
  The story you mention may indeed appear in an Oliver Sacks book (I've read several, and the story is familiar to me) but it doesn't seem to be in The Man Who Mistook His Wife for a Hat, according to the list in the article you linked.
  - FigBug 13y ago
    The book has 24 essays, they aren't all listed in the Wiki page.
- mantrax 13y ago
  Calling being unable to decide what to wear as "only being able to make rational decisions" is interesting, but it's not what's really happening. Every decision has a ROI. The line where ROI meets costs is your maximum budget for a decision, after which point it costs you more to remain in the decision phase that it costs you to follow the best of the few "imperfect" decisions you have in mind (one of them includes the decision to stop "deciding", do nothing, and move on - that's still deciding). That idea so far is rational. If you wear nothing you'll remain naked. If you don't eat you'll die. And most of all, being stuck deciding forever just about anything is not rational. So this person is unable to sense his "decision budget" and decides forever, while most people have facilities (impatience, impulsiveness in some degree, boredom etc.) to work around us being stuck forever after our decision budget is spent. Spending to get positive ROI is rational. Spending infinitely beyond that point is not.
maxerickson 13y ago
I guess we get hungry for a reason. (In that, at least in some sense, hunger is an irrational desire for food, it isn't reasoned out)
derefr 13y ago
I would think that this is the part that would need to be changed: > The utility of that is -inf. The utility of not opening the door is also -inf. Humans don't value anything--even their own continued existence--at inf, just at pretty average-levels-of-high numbers. We seem to get "intelligent" things done despite this. What's wrong with building an AI this way?
- waps 13y ago
  The problem is that the value of your own continued existence is Inf, not just because you don't want to die, but because you lose control after that fact, and thus all subsequent negative events are added to that cost. But aside from that, humans are not rational. Depending on your definition, humans aren't even intelligent. When it comes right down to it, it's pretty obvious what the human mind does to solve problems. It copies behavior. That's all it's capable of. It doesn't try to be rational, good, ... It is like DNA in that way (where 99.9999999999% of behavior simply comes from the parent(s)). People keep forgetting that evolution didn't create a "human", evolution is simply trying to duplicate itself. Your brain is nothing more or less than another trick to accomplish that. More detailed, here's what it does : 1) it looks at it's inputs, and models them : it finds how the feedback loop works between outputs and inputs (e.g. you move muscles 382, 182 and 138, your hand moves in the image of your eyes. That sort of thing and more complex) (technically it doesn't really do this. Your brain models the outside world as if it's part of your brain. That it models accurate prediction models is just a consequence of that) 2) it "shortcuts" the feedback loop. Essentially if neuron B always sees something happen 10 ms after neuron A sees it happen, they will form a connection and neuron B will report the event when neuron A starts seeing it's other event. If this means anything, it's a huge autocorrelation machine (with only positive offsets) 3) it copies behavior from it's predictive model to it's outputs. A human effectively tries to be the most "average" human in existence, based on his/her idea of what that is. It also copies (much less) from other events, like trees moving, animals, even just random signals that somehow get into your nervous system (some were put there on purpose). And lastly it copies from various internal sources that were put into our nervous system, like various clocks (I'm sure you can see why a clock would be useful to correlate things with), and chemical signals that get translated (like hunger, pain, or exitement, or ...). Keep in mind that I can split it up into functional things as much as I want. "Phase 1", "Phase 2", "Phase 3", but they don't really exist. The brain starts with a random signal and then does one operation continuously. That one operation implements 1,2 and 3 as a consequence of
stretchwithme 13y ago
I disagree that the rational actor would never do anything at all. He has needs for food, protection and shelter and these cannot be met without action. So he balances the low risk of a bear against the 100% chance of starving and finds it best to open the door.
partomniscient 13y ago
Contrived experiment is contrived.

by waps 13y ago

FigBug 13y ago

In "The Man Who mistook His Wife for a Hat" (http://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_Wife_for_a_Hat http://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_Wife_fo...), a book on mental disorders, there is a case of a man who can only make rational decisions. He is unable to function, he can't get past what to wear or what to eat in the morning. Even this simplest decisions take him hours.

taejo 13y ago
The story you mention may indeed appear in an Oliver Sacks book (I've read several, and the story is familiar to me) but it doesn't seem to be in The Man Who Mistook His Wife for a Hat, according to the list in the article you linked.
- FigBug 13y ago
  The book has 24 essays, they aren't all listed in the Wiki page.
mantrax 13y ago
Calling being unable to decide what to wear as "only being able to make rational decisions" is interesting, but it's not what's really happening. Every decision has a ROI. The line where ROI meets costs is your maximum budget for a decision, after which point it costs you more to remain in the decision phase that it costs you to follow the best of the few "imperfect" decisions you have in mind (one of them includes the decision to stop "deciding", do nothing, and move on - that's still deciding). That idea so far is rational. If you wear nothing you'll remain naked. If you don't eat you'll die. And most of all, being stuck deciding forever just about anything is not rational. So this person is unable to sense his "decision budget" and decides forever, while most people have facilities (impatience, impulsiveness in some degree, boredom etc.) to work around us being stuck forever after our decision budget is spent. Spending to get positive ROI is rational. Spending infinitely beyond that point is not.

maxerickson 13y ago

I guess we get hungry for a reason. (In that, at least in some sense, hunger is an irrational desire for food, it isn't reasoned out)

derefr 13y ago

I would think that this is the part that would need to be changed: > The utility of that is -inf. The utility of not opening the door is also -inf. Humans don't value anything--even their own continued existence--at inf, just at pretty average-levels-of-high numbers. We seem to get "intelligent" things done despite this. What's wrong with building an AI this way?

waps 13y ago
The problem is that the value of your own continued existence is Inf, not just because you don't want to die, but because you lose control after that fact, and thus all subsequent negative events are added to that cost. But aside from that, humans are not rational. Depending on your definition, humans aren't even intelligent. When it comes right down to it, it's pretty obvious what the human mind does to solve problems. It copies behavior. That's all it's capable of. It doesn't try to be rational, good, ... It is like DNA in that way (where 99.9999999999% of behavior simply comes from the parent(s)). People keep forgetting that evolution didn't create a "human", evolution is simply trying to duplicate itself. Your brain is nothing more or less than another trick to accomplish that. More detailed, here's what it does : 1) it looks at it's inputs, and models them : it finds how the feedback loop works between outputs and inputs (e.g. you move muscles 382, 182 and 138, your hand moves in the image of your eyes. That sort of thing and more complex) (technically it doesn't really do this. Your brain models the outside world as if it's part of your brain. That it models accurate prediction models is just a consequence of that) 2) it "shortcuts" the feedback loop. Essentially if neuron B always sees something happen 10 ms after neuron A sees it happen, they will form a connection and neuron B will report the event when neuron A starts seeing it's other event. If this means anything, it's a huge autocorrelation machine (with only positive offsets) 3) it copies behavior from it's predictive model to it's outputs. A human effectively tries to be the most "average" human in existence, based on his/her idea of what that is. It also copies (much less) from other events, like trees moving, animals, even just random signals that somehow get into your nervous system (some were put there on purpose). And lastly it copies from various internal sources that were put into our nervous system, like various clocks (I'm sure you can see why a clock would be useful to correlate things with), and chemical signals that get translated (like hunger, pain, or exitement, or ...). Keep in mind that I can split it up into functional things as much as I want. "Phase 1", "Phase 2", "Phase 3", but they don't really exist. The brain starts with a random signal and then does one operation continuously. That one operation implements 1,2 and 3 as a consequence of

stretchwithme 13y ago

I disagree that the rational actor would never do anything at all. He has needs for food, protection and shelter and these cannot be met without action. So he balances the low risk of a bear against the 100% chance of starving and finds it best to open the door.

partomniscient 13y ago

Contrived experiment is contrived.