4 ms·

What if it's impossible to develop an autonomous machine intelligence? I know that statement is equivalent to "what if it's impossible to develop heavier-than-a

What if it's impossible to develop an autonomous machine intelligence? I know that statement is equivalent to "what if it's impossible to develop heavier-than-air flight," and I think that we definitely should pursue AI research, but it is by no means guaranteed that we will end up with anything that behaves like a self-directed mind. I'm choosing my words here, because obviously a Singularity situation would not require a human-like intelligence. But as far as I know, neurologists and psychologists don't really know how human minds work, and computer scientists haven't built a computer mind that showed even a glimmer of "free will" or "self-awareness," if you'll pardon the terms. How can we take it for granted that we'll get there?

khafra 17y ago
The Singularity doesn't require computer sentience. Researchers at the Singularity Institute* refer to their goal as a "powerful optimization process."* All that's required is that it's better at general-purpose goal-seeking than humans; that would logically include the ability to wipe out the human race and tile the solar system with little smiley faces if we set its goal incorrectly. * http://www.singinst.org/ http://www.singinst.org/ * http://www.sl4.org/archive/0512/13006.html http://www.sl4.org/archive/0512/13006.html
- iron_ball 17y ago
  And that's roughly what I meant by "self-directed." Imagine a concrete goal: efficient fusion power, for instance. It's easy to define, easy to establish success metrics, and it's even easy to propose methods -- but success has eluded us for decades. A mind that could solve a problem like that would have to have intuition, lateral and parallel thinking, and creativity (or their machine equivalents; I'm open to an AI which might think utterly unlike a human). My point is that we don't know where such traits come from in humans, and thus have no idea how to even begin to attempt to replicate them in computers except in the crudest and most rule-based ways. Note: I would dearly like to be proven wrong, I'm just parroting things I've heard.
JesseAldridge 17y ago
Not "if" but "when". There's nothing magical about the human brain. It's just a bunch of neurons. Eventually we'll be able to replicate it.
pj 17y ago
It could be impossible to build autonomous machine intelligence. Depending on how you definie intelligence. Perhaps the intelligence of machines is a tangent curve approaching human intelligence. It is something in the future. We don't know for sure that it will happen, so by definition, it's an "opinion." All we have to go on are the trends we see around us now. Computers seem to be getting smarter and smarter, though it was a human that made them that way. "Impossible" itself is an opinion. The Japanese have commited $12 Billion toward working around the physical laws preventing us from building a space elevator. We can work around physical laws just like we can work around bugs in lower levels of the stack. But yes, there is a non-zero probability that it may not happen. I "believe" there is a greater non-zero probability that it will happen. It starts to get pretty philosophical. You could say, "Maybe it'll kill all the humans." Okay, that's a probability. Usually the implication of that is that killing the humans would be a bad thing, because we attribute negative thoughts towards death. Not all cultures do. If you think about life that reincarnates, then maybe intelligence is the life that thought refers to. Maybe it is a description of intelligences re-evolving. Maybe there have been intelligent species on this earth before with no fossil or material record to prove it. When you start to get to that level of thought about the topic, it all starts to break down and people disregard it. Because there is no "proof" we revert to what we can prove and what we have proof for -- for ourselves -- is what we feel inside: hunger, desire, thirst, etc. Those innate feelings usually overrule the intellectual ones. Those feelings of hunger drive the acceleration of technology and the acceleration of technology leads to smarter beings. Rewind to Einstein. He could have thought nuclear power was impossible. If he had described what he foresaw occurring as a result of the release of nuclear power, then many people would have said, "It's impossible to destroy a city and kill 100,000 people with a 5 ton bomb." Fat man was 10,200 lbs. We can destroy more now with less. Did Einstein stop his work, even knowing the potential? No. He said this kind of power didn't create new social problems, it only made the solutions to them more pressing. I think we could say the same thing here. We need to figure out how to exist on this planet together before we invent something that allows us to destroy all of us. From a more malthusian, perhaps, dark, perspective, the iRobot perspective, it may be a "smart" thing to erradicate humans. Another alternative may be that this thing we are building is smart enough to capture enough energy to vaporize the earth and it'll do that simply to answer a question. Maybe this thing we create will only be smart enough to cause total distruction, but not avoid it, or even know which of its actions will lead to it... The topic really raises a lot of questions. More questions than answers. I don't know how to answer them all. I have to have faith that humans will answer them correctly when faced with the questions. Hopefully they'll make the right choices.

khafra 17y ago

The Singularity doesn't require computer sentience. Researchers at the Singularity Institute* refer to their goal as a "powerful optimization process."* All that's required is that it's better at general-purpose goal-seeking than humans; that would logically include the ability to wipe out the human race and tile the solar system with little smiley faces if we set its goal incorrectly. * http://www.singinst.org/ http://www.singinst.org/ * http://www.sl4.org/archive/0512/13006.html http://www.sl4.org/archive/0512/13006.html

iron_ball 17y ago
And that's roughly what I meant by "self-directed." Imagine a concrete goal: efficient fusion power, for instance. It's easy to define, easy to establish success metrics, and it's even easy to propose methods -- but success has eluded us for decades. A mind that could solve a problem like that would have to have intuition, lateral and parallel thinking, and creativity (or their machine equivalents; I'm open to an AI which might think utterly unlike a human). My point is that we don't know where such traits come from in humans, and thus have no idea how to even begin to attempt to replicate them in computers except in the crudest and most rule-based ways. Note: I would dearly like to be proven wrong, I'm just parroting things I've heard.

JesseAldridge 17y ago

Not "if" but "when". There's nothing magical about the human brain. It's just a bunch of neurons. Eventually we'll be able to replicate it.

pj 17y ago

It could be impossible to build autonomous machine intelligence. Depending on how you definie intelligence. Perhaps the intelligence of machines is a tangent curve approaching human intelligence. It is something in the future. We don't know for sure that it will happen, so by definition, it's an "opinion." All we have to go on are the trends we see around us now. Computers seem to be getting smarter and smarter, though it was a human that made them that way. "Impossible" itself is an opinion. The Japanese have commited $12 Billion toward working around the physical laws preventing us from building a space elevator. We can work around physical laws just like we can work around bugs in lower levels of the stack. But yes, there is a non-zero probability that it may not happen. I "believe" there is a greater non-zero probability that it will happen. It starts to get pretty philosophical. You could say, "Maybe it'll kill all the humans." Okay, that's a probability. Usually the implication of that is that killing the humans would be a bad thing, because we attribute negative thoughts towards death. Not all cultures do. If you think about life that reincarnates, then maybe intelligence is the life that thought refers to. Maybe it is a description of intelligences re-evolving. Maybe there have been intelligent species on this earth before with no fossil or material record to prove it. When you start to get to that level of thought about the topic, it all starts to break down and people disregard it. Because there is no "proof" we revert to what we can prove and what we have proof for -- for ourselves -- is what we feel inside: hunger, desire, thirst, etc. Those innate feelings usually overrule the intellectual ones. Those feelings of hunger drive the acceleration of technology and the acceleration of technology leads to smarter beings. Rewind to Einstein. He could have thought nuclear power was impossible. If he had described what he foresaw occurring as a result of the release of nuclear power, then many people would have said, "It's impossible to destroy a city and kill 100,000 people with a 5 ton bomb." Fat man was 10,200 lbs. We can destroy more now with less. Did Einstein stop his work, even knowing the potential? No. He said this kind of power didn't create new social problems, it only made the solutions to them more pressing. I think we could say the same thing here. We need to figure out how to exist on this planet together before we invent something that allows us to destroy all of us. From a more malthusian, perhaps, dark, perspective, the iRobot perspective, it may be a "smart" thing to erradicate humans. Another alternative may be that this thing we are building is smart enough to capture enough energy to vaporize the earth and it'll do that simply to answer a question. Maybe this thing we create will only be smart enough to cause total distruction, but not avoid it, or even know which of its actions will lead to it... The topic really raises a lot of questions. More questions than answers. I don't know how to answer them all. I have to have faith that humans will answer them correctly when faced with the questions. Hopefully they'll make the right choices.