4 ms·

Great article. I've always found the concept of singularity naive. Greatest discoveries haven't been done by the most intelligent persons. Science isn't just

Great article. I've always found the concept of singularity naive. Greatest discoveries haven't been done by the most intelligent persons. Science isn't just a question of intelligence or will, it's also a question of timing, luck and duration. Everything has a limit. Ants have limits. We have limits. And the supposed "all mighty AI" will have limits if it ever exists.

harshreality 14y ago
And yet, what are ants to us? Limits of an AI may be incomprehensible to us, and the point of singularity is not that intelligence might increase without bound, but that it will reach our level and appear be increasing at an exponentially beyond us. (Any increase in intelligence beyond our level may be obvious but unquantifiable, so the question of exponential vs simply "beyond us" may be unanswerable.) Logistics curves are a favorite concept on HN relating to resource usage, so imagine that intelligence follows a logistic curve, and humanity is way above ants, but still very close to the bottom. Once an AI gains the ability to iteratively improve its own mind, first software, and then hardware, it could accelerate up the curve and the separation between our intelligence and AI, if measured, would be approximately exponential until it's so far ahead of us it doesn't matter whether there's a hard limit (logistic curve) or not (true exponential curve).
- shin_lao 14y ago
  Sorry, I think my point wasn't properly explained. My point is that intelligence isn't everything. An AI may become more "intelligent" than a human being, but it doesn't mean any singularity will happen. It does not mean it can self-improve. It does not mean it can find solutions we did not think of. It's a bit hyperbolic to say "we don't know what greater intelligence can do" and conclude "it will therefore do incredible things". And to bounce on your answer, are we more intelligent than ants? Does it even make sense to say "we're more intelligent than ants"? Our differences with ants go far beyond intelligence.
  - harshreality 14y ago
    An AI would be able to self-improve its software (unless artificially limited, and it's not clear to me whether that would be effective), and with enough initial robotic and sensory capabilities it would be able to self-improve its hardware, both computational aspects and robotic aspects. The only reasons it wouldn't are if it didn't put effort into it (too busy watching Lost), or if intelligence cannot really be improved. Humans might do the equivalent of rewriting software through thinking. But it's a more limited and slow process than could be accomplished with full read/write access to software running an AI that's completely malleable and rewritable in moments. There's plenty of evidence that intelligence can be improved. Working memory is important. We have very limited working memory. We are frequently distracted by emotions. If an AI can operate at roughly equivalent speeds with greater working memory, and if it's not distracted by emotions, it may miss some of the "finer" points of life, but I can't imagine it not leapfrogging ahead of meager human progress. Maybe it's not possible. Like the thing at the beginning of Fire Upon the Deep, if it is possible, we won't really know if it is until it exists. If it's got the temperment of a class 2 perversion or the Blight, we're toast. If it's a Power that finds us amusing or irrelevant, we might be okay.
marvin 14y ago
I posted this link somewhere else, but it's such a good read that I'll post it again. http://singularity.org/research/ http://singularity.org/research/ Even if you disagree with the ideas of what general artificial intelligence could do to the human existence on Earth and elsewhere, it is justified to spend some effort thinking about it. The potential consequences to human existence are all-encompassing. So when something is deemed a potential existential risk, it's worth thinking about. If the current development in computing continues, I believe that superhuman artificial intelligence is inevitable. And that if we don't prepare for it properly, we might screw it up in a historical way.

harshreality 14y ago

And yet, what are ants to us? Limits of an AI may be incomprehensible to us, and the point of singularity is not that intelligence might increase without bound, but that it will reach our level and appear be increasing at an exponentially beyond us. (Any increase in intelligence beyond our level may be obvious but unquantifiable, so the question of exponential vs simply "beyond us" may be unanswerable.) Logistics curves are a favorite concept on HN relating to resource usage, so imagine that intelligence follows a logistic curve, and humanity is way above ants, but still very close to the bottom. Once an AI gains the ability to iteratively improve its own mind, first software, and then hardware, it could accelerate up the curve and the separation between our intelligence and AI, if measured, would be approximately exponential until it's so far ahead of us it doesn't matter whether there's a hard limit (logistic curve) or not (true exponential curve).

shin_lao 14y ago
Sorry, I think my point wasn't properly explained. My point is that intelligence isn't everything. An AI may become more "intelligent" than a human being, but it doesn't mean any singularity will happen. It does not mean it can self-improve. It does not mean it can find solutions we did not think of. It's a bit hyperbolic to say "we don't know what greater intelligence can do" and conclude "it will therefore do incredible things". And to bounce on your answer, are we more intelligent than ants? Does it even make sense to say "we're more intelligent than ants"? Our differences with ants go far beyond intelligence.
- harshreality 14y ago
  An AI would be able to self-improve its software (unless artificially limited, and it's not clear to me whether that would be effective), and with enough initial robotic and sensory capabilities it would be able to self-improve its hardware, both computational aspects and robotic aspects. The only reasons it wouldn't are if it didn't put effort into it (too busy watching Lost), or if intelligence cannot really be improved. Humans might do the equivalent of rewriting software through thinking. But it's a more limited and slow process than could be accomplished with full read/write access to software running an AI that's completely malleable and rewritable in moments. There's plenty of evidence that intelligence can be improved. Working memory is important. We have very limited working memory. We are frequently distracted by emotions. If an AI can operate at roughly equivalent speeds with greater working memory, and if it's not distracted by emotions, it may miss some of the "finer" points of life, but I can't imagine it not leapfrogging ahead of meager human progress. Maybe it's not possible. Like the thing at the beginning of Fire Upon the Deep, if it is possible, we won't really know if it is until it exists. If it's got the temperment of a class 2 perversion or the Blight, we're toast. If it's a Power that finds us amusing or irrelevant, we might be okay.

marvin 14y ago

I posted this link somewhere else, but it's such a good read that I'll post it again. http://singularity.org/research/ http://singularity.org/research/ Even if you disagree with the ideas of what general artificial intelligence could do to the human existence on Earth and elsewhere, it is justified to spend some effort thinking about it. The potential consequences to human existence are all-encompassing. So when something is deemed a potential existential risk, it's worth thinking about. If the current development in computing continues, I believe that superhuman artificial intelligence is inevitable. And that if we don't prepare for it properly, we might screw it up in a historical way.