3 ms·

> I think the reason for this is that our brains just generally don't operate on a rational basis because it's not practical for humans. We have to rely on pref

> I think the reason for this is that our brains just generally don't operate on a rational basis because it's not practical for humans. We have to rely on preformed perspectives when it comes to certain broad perspectives. I think there is a more fundamental reason, ie, that there is no complete rational explanation of reality. What we do have is partial rational explanations which work well in some specific contexts, but are not absolute: they break down if you change context. See for example relativity/quantum mechanics; even the hardest of hard sciences doesn't have a unified, complete theory. That's why AI based on logical reasoning didn't work, and will never work IMHO. Rationality is a tool that intelligent creatures can use - the most advanced tool, probably - but it's not enough to live in the real world. Some of the cognitive shortcuts we use are "only" necessary to cut decision time, but many are fundamentally necessary for adaptation to our environment.

sooheon 8y ago
> that there is no complete rational explanation of reality ... that fits inside a current day human's brain. I see no reason to believe this is true fundamentally.
- tomhoward 8y ago
  They were referring to present time, not a hypothetical future of more complete explanations of nature and more powerful human brains for conceiving and storing them. That said, there is a reason why it may not be possible: human consciousness can only exist at a higher level than the most fundamental level of nature, and on that basis, consciousness may never be able to perceive enough of the fundamentals of nature to be able to formulate a complete explanation of it. However it's not really relevant to the topic at hand. I'm sure humans will always use cognitive shortcuts like ideologies in order to navigate our daily experience, for reasons of speed and efficiency.
- danmaz74 8y ago
  Of course we can't prove anything about possible future non-human brains, but try to think about it this way. Is there a limit to the number of consequences that a single action can have? If you think about all possible future consequences, then the answer is clearly no. Using an example from another comment, opening a closed door could bring you to your death, which has infinite consequences for your (potentially future) family. Therefore, if rationally analysing each possible consequence takes any non-zero amount of time, deciding any action rationally would take an infinite time. You could try to rationally decide what is the maximum amount of time you can spend on each choice, but deciding that amount rationally would require infinite time. This isn't a proof, but this consideration comes from spending a lot of time thinking about rationality and AI.
  - sooheon 8y ago
    I think you're conflating two thing: solvability and rationality. Your example shows that nothing may be completely solvable, because the cost of calculation is too high. It says nothing about the universe being fundamentally irrational.
    - danmaz74 8y ago
      The problem I had in mind is if it's possible, for a human, to decide what to do in each situation based only on rationality. I wouldn't know how to define if the universe itself is rational, in an absolute/abstract sense.
candiodari 8y ago
This was at one point (in the AI winter before the 2006, I believe) referred to as the "starving at the door" problem. The "problem" was that a rational robot (or rational human) needs to decide to open a door or not. He can't just stay in front of the door, because eventually he'll die. So the "cost" of waiting is small, but accumulating. And of course, we know this is stupid. He also cannot open the door. There might be a bear behind the door, which might kill him. Small, but nonzero chance of that happening. Cost of death is inf (because it fails all other goals, it must be a higher cost than the sum of failing everything, therefore effectively it's infinite). So a rational robot/human will never open the door. Rather he'll die in front of it, from exhaustion. The fun thing is that you actually see this behavior in rule based reasoning systems. They will refuse to do anything at all, because it might kill them (even if they technically don't know what death is this still happens. They can still add together the costs of failing everything). You have to go in and manually remove these cases if you want things to work. So yes, you need a belief system. Even a purely rational decision making systems needs a belief system. To solve this problem it needs to believe it will never die (or at least that a sufficiently small chance of death means death never occurs). The fun thing is, this is a true belief: it's demonstrably wrong. In a practical system there will be a lot more beliefs than just this one.
- danmaz74 8y ago
  Thanks, "belief systems" is one way to describe those partial theories which work on a practical level. Thing is, you need many, and they're contradictory. I see them as similar to linearisation of complex functions: they work well in a neighborhood, but not outside of it. The real function we'd like to reason about has infinite dimensions, is can't really be measured... "All models are wrong, but some are useful" George E.P. Box
- leereeves 8y ago
  > So a rational robot/human will never open the door. If the door leads from outside to inside, there's also a small chance the robot might die if it doesn't go inside (a passing car might hit it, anti-AI vandals might destroy it, etc). It's probably safer inside. So perhaps we could say that a rational robot/human will never go outside. Interestingly, some people do exhibit this behavior (agoraphobia) and it's generally considered anything but rational. The problem for the AI can be fixed, without a belief system, with a lower cost for danger. Infinite cost is too high because it makes probability irrelevant. A finite but very high "cost of death" would still avoid likely death without becoming paralyzed by the possibility of unlikely events. We sometimes see people deciding that a cause is worth risking, even knowingly giving, their life for, proving that even humans don't weigh the cost of death above all other activities, so it's hard to see why a nonsentient AI should do so.

sooheon 8y ago

> that there is no complete rational explanation of reality ... that fits inside a current day human's brain. I see no reason to believe this is true fundamentally.

tomhoward 8y ago
They were referring to present time, not a hypothetical future of more complete explanations of nature and more powerful human brains for conceiving and storing them. That said, there is a reason why it may not be possible: human consciousness can only exist at a higher level than the most fundamental level of nature, and on that basis, consciousness may never be able to perceive enough of the fundamentals of nature to be able to formulate a complete explanation of it. However it's not really relevant to the topic at hand. I'm sure humans will always use cognitive shortcuts like ideologies in order to navigate our daily experience, for reasons of speed and efficiency.
danmaz74 8y ago
Of course we can't prove anything about possible future non-human brains, but try to think about it this way. Is there a limit to the number of consequences that a single action can have? If you think about all possible future consequences, then the answer is clearly no. Using an example from another comment, opening a closed door could bring you to your death, which has infinite consequences for your (potentially future) family. Therefore, if rationally analysing each possible consequence takes any non-zero amount of time, deciding any action rationally would take an infinite time. You could try to rationally decide what is the maximum amount of time you can spend on each choice, but deciding that amount rationally would require infinite time. This isn't a proof, but this consideration comes from spending a lot of time thinking about rationality and AI.
- sooheon 8y ago
  I think you're conflating two thing: solvability and rationality. Your example shows that nothing may be completely solvable, because the cost of calculation is too high. It says nothing about the universe being fundamentally irrational.
  - danmaz74 8y ago
    The problem I had in mind is if it's possible, for a human, to decide what to do in each situation based only on rationality. I wouldn't know how to define if the universe itself is rational, in an absolute/abstract sense.

candiodari 8y ago

This was at one point (in the AI winter before the 2006, I believe) referred to as the "starving at the door" problem. The "problem" was that a rational robot (or rational human) needs to decide to open a door or not. He can't just stay in front of the door, because eventually he'll die. So the "cost" of waiting is small, but accumulating. And of course, we know this is stupid. He also cannot open the door. There might be a bear behind the door, which might kill him. Small, but nonzero chance of that happening. Cost of death is inf (because it fails all other goals, it must be a higher cost than the sum of failing everything, therefore effectively it's infinite). So a rational robot/human will never open the door. Rather he'll die in front of it, from exhaustion. The fun thing is that you actually see this behavior in rule based reasoning systems. They will refuse to do anything at all, because it might kill them (even if they technically don't know what death is this still happens. They can still add together the costs of failing everything). You have to go in and manually remove these cases if you want things to work. So yes, you need a belief system. Even a purely rational decision making systems needs a belief system. To solve this problem it needs to believe it will never die (or at least that a sufficiently small chance of death means death never occurs). The fun thing is, this is a true belief: it's demonstrably wrong. In a practical system there will be a lot more beliefs than just this one.

danmaz74 8y ago
Thanks, "belief systems" is one way to describe those partial theories which work on a practical level. Thing is, you need many, and they're contradictory. I see them as similar to linearisation of complex functions: they work well in a neighborhood, but not outside of it. The real function we'd like to reason about has infinite dimensions, is can't really be measured... "All models are wrong, but some are useful" George E.P. Box
leereeves 8y ago
> So a rational robot/human will never open the door. If the door leads from outside to inside, there's also a small chance the robot might die if it doesn't go inside (a passing car might hit it, anti-AI vandals might destroy it, etc). It's probably safer inside. So perhaps we could say that a rational robot/human will never go outside. Interestingly, some people do exhibit this behavior (agoraphobia) and it's generally considered anything but rational. The problem for the AI can be fixed, without a belief system, with a lower cost for danger. Infinite cost is too high because it makes probability irrelevant. A finite but very high "cost of death" would still avoid likely death without becoming paralyzed by the possibility of unlikely events. We sometimes see people deciding that a cause is worth risking, even knowingly giving, their life for, proving that even humans don't weigh the cost of death above all other activities, so it's hard to see why a nonsentient AI should do so.