3 ms·

For a rational agent, a decision is good or bad based on it's preferences, which are described mathematically as part of the definition of the rational agent.

For a rational agent, a decision is good or bad based on it's preferences, which are described mathematically as part of the definition of the rational agent. So I don't believe it's tautological. I do agree with your statement that intelligence is difficult to describe. And for the record I never claimed the rational agent approach will achieve "true machine intelligence". Frankly, I don't even know what "true" machine intelligence is or would be. What I am very interested in however is how to create an AI that is capable of fooling a person into thinking it's intelligent, in other words an AI that can pass the Turing test. For that purpose I find the concept of a rational agent to be very helpful, in that it provides a solid foundation to work from. I'm also not worried about Godel's theorem at all. The statements that are undecidable are very obscure and basically never come into play in human decision making. I think Godel's result has very little practical application, if it has any at all; I certainly don't think it's relevant for AI. I agree that choosing a logical framework to work in presents many problems, but I don't agree that you have to disprove Godel's theorem to create an AI that can think outside the box.

ac500 14y ago
I also don't think you have to disprove Godel's theorem to create an AI that thinks outside the box, however, I believe you would have to for it to be a rational agent. A rational agent by definition operates strictly within the domain of a formal system. Thus by definition, a rational agent is in direct conflict with the notion of "thinking outside the box." Of course any simulation is implemented in some formal system, but that's not what I'm saying. I'm saying the inherent problem with a rational agent is that they maximize utility within a formal system -- that is what they do, that is their entire purpose. Once we start talking about rational agents that can "think outside the box", or think beyond the formal system in which they're defined, we're no longer talking about rational agents. I agree with your belief that we may some day build AIs that think outside the box, I just don't see how it can be a rational agent. I think it will be much closer to a neural network, or probabilistic reasoning model, or some extremely "organic" or "fluid" device from which intelligent behavior organically emerges. Because if intelligence emerges from a strictly formalized framework, then Godel's theorems prove some very crippling limitations on what's possible within that fixed framework. FYI there is definitely a connection between Godel's theorem and intelligence, though it may not be immediately obvious. I would highly recommend this as fun learning material: http://ocw.mit.edu/high-school/courses/godel-escher-bach/ http://ocw.mit.edu/high-school/courses/godel-escher-bach/
- solistice 14y ago
  Wait, wasn't a rational agent simply an agent that optimized a certain result given a set of knowledge about it and it's environment? I mean in engineering and physics, you do try to build the perfect something, but you also realize you will never ever build the perfect something. You'll never build the perfect computer, the perfect op amp, nor the perfect rational agent, just as you'll never find the biggest number, because there's no upper bounds on these kind of things.
  - ac500 14y ago
    Try to implement a rational agent (the AI or mathematically formalized variety) in software, and you'll see what I mean. You have to define some utility function precisely, and some algorithm to maximize it. The problem with this is it won't really produce a "general intelligence" that can think outside the box, because it will always be maximizing some utility function defined in some rigid formal system. In other words, it will be completely unable to "understand" things outside of the formal system in which you define it.
Evbn 14y ago
Thank you for knocking down the Gödel canard. People who say humans are not similar able have a vague faith that humans are far smarter than they really are. The existence of "Uhprovably true statements" is less of a weakness in logic's ability to organize the universe, and more that there are always less meaningful yet constructible statements, like "This statement is not provable."
- ac500 14y ago
  > "People who say humans are not similar able have a vague faith that humans are far smarter than they really are." Prenote: I apologize in advance if my entire response is off-topic, because my English parser was completely unable to take anything meaningful from this sentence. When you have a system of axioms, both consistent and complete, which proves all useful properties of arithmetic and algebra, let us know. Whether or not you are able to understand the deepness of connection between Godel's theorems, formal systems, self-similarity, and intelligence, you can't deny that the more statistical / 'organic' AIs are overwhelmingly more successful than rigidly logical ones. For example, try implementing voice recognition analytically, and let us know your results (have fun with that). The point is that although some people would like to believe otherwise, human brains (and nonhuman brains) are NOT logical reasoning machines. Humans are not rational agents. It still surprises me how many people are desperately grasping to rational agents and similar frameworks to try to build general intelligences entirely from logic. I point out Godel's theorems as one type of insight into why this road will lead you nowhere. You're free to pursue it as much as you like, just as animals are free to run in circles chasing their tail. You can't build an AI capable of thinking outside the box, by specifically building that AI in the box of a particular logical framework. Logic itself has limitations. Logic is powerful, but not entirely in of itself -- logic is powerful as a tool extended by the mind of its creator(s). The more meaningful question in AI, is how do we create logical concepts to begin with?

ac500 14y ago

I also don't think you have to disprove Godel's theorem to create an AI that thinks outside the box, however, I believe you would have to for it to be a rational agent. A rational agent by definition operates strictly within the domain of a formal system. Thus by definition, a rational agent is in direct conflict with the notion of "thinking outside the box." Of course any simulation is implemented in some formal system, but that's not what I'm saying. I'm saying the inherent problem with a rational agent is that they maximize utility within a formal system -- that is what they do, that is their entire purpose. Once we start talking about rational agents that can "think outside the box", or think beyond the formal system in which they're defined, we're no longer talking about rational agents. I agree with your belief that we may some day build AIs that think outside the box, I just don't see how it can be a rational agent. I think it will be much closer to a neural network, or probabilistic reasoning model, or some extremely "organic" or "fluid" device from which intelligent behavior organically emerges. Because if intelligence emerges from a strictly formalized framework, then Godel's theorems prove some very crippling limitations on what's possible within that fixed framework. FYI there is definitely a connection between Godel's theorem and intelligence, though it may not be immediately obvious. I would highly recommend this as fun learning material: http://ocw.mit.edu/high-school/courses/godel-escher-bach/ http://ocw.mit.edu/high-school/courses/godel-escher-bach/

solistice 14y ago
Wait, wasn't a rational agent simply an agent that optimized a certain result given a set of knowledge about it and it's environment? I mean in engineering and physics, you do try to build the perfect something, but you also realize you will never ever build the perfect something. You'll never build the perfect computer, the perfect op amp, nor the perfect rational agent, just as you'll never find the biggest number, because there's no upper bounds on these kind of things.
- ac500 14y ago
  Try to implement a rational agent (the AI or mathematically formalized variety) in software, and you'll see what I mean. You have to define some utility function precisely, and some algorithm to maximize it. The problem with this is it won't really produce a "general intelligence" that can think outside the box, because it will always be maximizing some utility function defined in some rigid formal system. In other words, it will be completely unable to "understand" things outside of the formal system in which you define it.

Evbn 14y ago

Thank you for knocking down the Gödel canard. People who say humans are not similar able have a vague faith that humans are far smarter than they really are. The existence of "Uhprovably true statements" is less of a weakness in logic's ability to organize the universe, and more that there are always less meaningful yet constructible statements, like "This statement is not provable."

ac500 14y ago
> "People who say humans are not similar able have a vague faith that humans are far smarter than they really are." Prenote: I apologize in advance if my entire response is off-topic, because my English parser was completely unable to take anything meaningful from this sentence. When you have a system of axioms, both consistent and complete, which proves all useful properties of arithmetic and algebra, let us know. Whether or not you are able to understand the deepness of connection between Godel's theorems, formal systems, self-similarity, and intelligence, you can't deny that the more statistical / 'organic' AIs are overwhelmingly more successful than rigidly logical ones. For example, try implementing voice recognition analytically, and let us know your results (have fun with that). The point is that although some people would like to believe otherwise, human brains (and nonhuman brains) are NOT logical reasoning machines. Humans are not rational agents. It still surprises me how many people are desperately grasping to rational agents and similar frameworks to try to build general intelligences entirely from logic. I point out Godel's theorems as one type of insight into why this road will lead you nowhere. You're free to pursue it as much as you like, just as animals are free to run in circles chasing their tail. You can't build an AI capable of thinking outside the box, by specifically building that AI in the box of a particular logical framework. Logic itself has limitations. Logic is powerful, but not entirely in of itself -- logic is powerful as a tool extended by the mind of its creator(s). The more meaningful question in AI, is how do we create logical concepts to begin with?