2 ms·

>Do you know of any examples of intelligent beings that don't have any motivations and drives? My computer, for certain definitions of intelligent. > Intellig

>Do you know of any examples of intelligent beings that don't have any motivations and drives? My computer, for certain definitions of intelligent. > Intelligence doesn't just mean cogitating in a vacuum; it means taking information in from the world, and doing things that have effects in the world. I agree with this; but its behavior does not have to be self-directed to be intelligent. Again, computers behave quite intelligently in certain constrained areas, yet its behavior is completely driven by a human operator. There is no reason a fully general AI must be self-directed based on what we would call drives. >So an AI has to at least have the motivation and drive to take in information and do things with it I don't see this to be true either. Its (supposed) neural network could be modified externally without any self-direction whatsoever. An intelligent process does not have to look like a simulation of ourselves. The word "being" perhaps is the stumbling point here. Perhaps it is true that something considered a "being" would necessarily require a certain level of self-direction. But even in that case I don't see it being possible for a being who was, say, programmed to enjoy absorbing knowledge to necessarily have any self-preservation instinct, or any drives whatsoever outside of knowledge-gathering. All the "ghosts in the machine" nonsense is pure science fiction. I don't think there is any programming error that could turn an intended knowledge-loving machine into a self-preserving amoral humanity killer. The architecture of the two would be vastly different.

pdonis 13y ago
for certain definitions of intelligent Yes, but I would argue that those definitions are not really relevant to this discussion. You say... behavior does not have to be self-directed to be intelligent ...which is true, but the whole point of AI is to get to a point where computers are self-directed; where we don't have to laboriously tell the computer what to do; we just give it a general goal statement and it figures out how to accomplish it. If we have to continuously intervene to get it to do what we want, what's the point? We have that now. So this... Its (supposed) neural network could be modified externally without any self-direction whatsoever. ...is also not really relevant, because the whole point is to develop AI's that can modify their own neural networks (or whatever internal structures they end up having) as an ongoing process, the way humans do. (Btw, one of the reasons I keep saying this is "the whole point" is that developing such AI's would confer a huge competitive advantage, compared to the "intelligent" machines we have now, that only exhibit "intelligent" behavior with continuous human intervention. So it's not realistic to limit discussion to the latter kind of machines; even if you personally don't want to take the next step, somebody else will.) The word "being" perhaps is the stumbling point here. No, I think it's the word "intelligent". See above. I don't think there is any programming error that could turn an intended knowledge-loving machine into a self-preserving amoral humanity killer. I don't think you're trying hard enough to imagine what effects a programming error could have. Have you read any of Eliezer Yudkowsky's articles on the Friendly AI problem? The architecture of the two would be vastly different. Why would this have to be the case? Human beings implement both behaviors quite handily on the same architecture.
- hackinthebochs 13y ago
  I have a feeling our disagreement is largely one of terminology. >the whole point of AI is to get to a point where computers are self-directed; where we don't have to laboriously tell the computer what to do; There is much in between laboriously tell it what to do and having a self-directed entity traipsing in and out of computer networks. A system that is smart enough to figure out how to accomplish a high level goal on its own doesn't have to be a self-directed entity. It just needs to be infused with enough real-world knowledge that the supposed optimization problem has a solution. >because the whole point is to develop AI's that can modify their own neural networks (or whatever internal structures they end up having) as an ongoing process, the way humans do. I think you give us too much credit. We may guide our learning processes, but the actual modification of our neural networks is completely out of our control. The distinction here may seem useless, but in this case it is important. A supposed unself-directed-but-intelligent being would simply not have the ability to direct its learning processes. We would still have to bootstrap its learning algorithms on a particular dataset to increase its knowledge base. But this isn't contradictory to general AI, nor would it be useless. In fact, I would say the only thing we would lose is the warm fuzzies that we created "life". It's still just as useful to us if we're in control of its growth.
  - pdonis 13y ago
    A system that is smart enough to figure out how to accomplish a high level goal on its own doesn't have to be a self-directed entity. But a system that can do this and is a self-directed entity provides, as I said, a competitive advantage over a system that can do this but isn't self-directed. So there will be an incentive for people to make the latter kind of system into the former kind. We may guide our learning processes, but the actual modification of our neural networks is completely out of our control. As you state it, this is false, because guiding our learning processes is controlling at least some aspects of the modification of our neural networks. But I agree that our control over the actual modification of our neural networks is extremely coarse; most aspects of it are out of our control. It's still just as useful to us if we're in control of its growth. No, it isn't, because if we're in control of its growth, its growth is limited by our mental capacities. An AI which can control its own growth is only limited by its own mental capacities, which could exceed ours. Since one of the biggest limitations on human progress is limited human mental capacity, an AI which can exceed that limit will be highly desirable. However, the price of that desirable thing is that, since by definition the AI's mental capacity exceeds that of humans, humans can no longer reliably exert control over it.

pdonis 13y ago

for certain definitions of intelligent Yes, but I would argue that those definitions are not really relevant to this discussion. You say... behavior does not have to be self-directed to be intelligent ...which is true, but the whole point of AI is to get to a point where computers are self-directed; where we don't have to laboriously tell the computer what to do; we just give it a general goal statement and it figures out how to accomplish it. If we have to continuously intervene to get it to do what we want, what's the point? We have that now. So this... Its (supposed) neural network could be modified externally without any self-direction whatsoever. ...is also not really relevant, because the whole point is to develop AI's that can modify their own neural networks (or whatever internal structures they end up having) as an ongoing process, the way humans do. (Btw, one of the reasons I keep saying this is "the whole point" is that developing such AI's would confer a huge competitive advantage, compared to the "intelligent" machines we have now, that only exhibit "intelligent" behavior with continuous human intervention. So it's not realistic to limit discussion to the latter kind of machines; even if you personally don't want to take the next step, somebody else will.) The word "being" perhaps is the stumbling point here. No, I think it's the word "intelligent". See above. I don't think there is any programming error that could turn an intended knowledge-loving machine into a self-preserving amoral humanity killer. I don't think you're trying hard enough to imagine what effects a programming error could have. Have you read any of Eliezer Yudkowsky's articles on the Friendly AI problem? The architecture of the two would be vastly different. Why would this have to be the case? Human beings implement both behaviors quite handily on the same architecture.

hackinthebochs 13y ago
I have a feeling our disagreement is largely one of terminology. >the whole point of AI is to get to a point where computers are self-directed; where we don't have to laboriously tell the computer what to do; There is much in between laboriously tell it what to do and having a self-directed entity traipsing in and out of computer networks. A system that is smart enough to figure out how to accomplish a high level goal on its own doesn't have to be a self-directed entity. It just needs to be infused with enough real-world knowledge that the supposed optimization problem has a solution. >because the whole point is to develop AI's that can modify their own neural networks (or whatever internal structures they end up having) as an ongoing process, the way humans do. I think you give us too much credit. We may guide our learning processes, but the actual modification of our neural networks is completely out of our control. The distinction here may seem useless, but in this case it is important. A supposed unself-directed-but-intelligent being would simply not have the ability to direct its learning processes. We would still have to bootstrap its learning algorithms on a particular dataset to increase its knowledge base. But this isn't contradictory to general AI, nor would it be useless. In fact, I would say the only thing we would lose is the warm fuzzies that we created "life". It's still just as useful to us if we're in control of its growth.
- pdonis 13y ago
  A system that is smart enough to figure out how to accomplish a high level goal on its own doesn't have to be a self-directed entity. But a system that can do this and is a self-directed entity provides, as I said, a competitive advantage over a system that can do this but isn't self-directed. So there will be an incentive for people to make the latter kind of system into the former kind. We may guide our learning processes, but the actual modification of our neural networks is completely out of our control. As you state it, this is false, because guiding our learning processes is controlling at least some aspects of the modification of our neural networks. But I agree that our control over the actual modification of our neural networks is extremely coarse; most aspects of it are out of our control. It's still just as useful to us if we're in control of its growth. No, it isn't, because if we're in control of its growth, its growth is limited by our mental capacities. An AI which can control its own growth is only limited by its own mental capacities, which could exceed ours. Since one of the biggest limitations on human progress is limited human mental capacity, an AI which can exceed that limit will be highly desirable. However, the price of that desirable thing is that, since by definition the AI's mental capacity exceeds that of humans, humans can no longer reliably exert control over it.