4 ms·

And yet, what are ants to us? Limits of an AI may be incomprehensible to us, and the point of singularity is not that intelligence might increase without bound

And yet, what are ants to us? Limits of an AI may be incomprehensible to us, and the point of singularity is not that intelligence might increase without bound, but that it will reach our level and appear be increasing at an exponentially beyond us. (Any increase in intelligence beyond our level may be obvious but unquantifiable, so the question of exponential vs simply "beyond us" may be unanswerable.) Logistics curves are a favorite concept on HN relating to resource usage, so imagine that intelligence follows a logistic curve, and humanity is way above ants, but still very close to the bottom. Once an AI gains the ability to iteratively improve its own mind, first software, and then hardware, it could accelerate up the curve and the separation between our intelligence and AI, if measured, would be approximately exponential until it's so far ahead of us it doesn't matter whether there's a hard limit (logistic curve) or not (true exponential curve).

shin_lao 14y ago
Sorry, I think my point wasn't properly explained. My point is that intelligence isn't everything. An AI may become more "intelligent" than a human being, but it doesn't mean any singularity will happen. It does not mean it can self-improve. It does not mean it can find solutions we did not think of. It's a bit hyperbolic to say "we don't know what greater intelligence can do" and conclude "it will therefore do incredible things". And to bounce on your answer, are we more intelligent than ants? Does it even make sense to say "we're more intelligent than ants"? Our differences with ants go far beyond intelligence.
- harshreality 14y ago
  An AI would be able to self-improve its software (unless artificially limited, and it's not clear to me whether that would be effective), and with enough initial robotic and sensory capabilities it would be able to self-improve its hardware, both computational aspects and robotic aspects. The only reasons it wouldn't are if it didn't put effort into it (too busy watching Lost), or if intelligence cannot really be improved. Humans might do the equivalent of rewriting software through thinking. But it's a more limited and slow process than could be accomplished with full read/write access to software running an AI that's completely malleable and rewritable in moments. There's plenty of evidence that intelligence can be improved. Working memory is important. We have very limited working memory. We are frequently distracted by emotions. If an AI can operate at roughly equivalent speeds with greater working memory, and if it's not distracted by emotions, it may miss some of the "finer" points of life, but I can't imagine it not leapfrogging ahead of meager human progress. Maybe it's not possible. Like the thing at the beginning of Fire Upon the Deep, if it is possible, we won't really know if it is until it exists. If it's got the temperment of a class 2 perversion or the Blight, we're toast. If it's a Power that finds us amusing or irrelevant, we might be okay.

shin_lao 14y ago

Sorry, I think my point wasn't properly explained. My point is that intelligence isn't everything. An AI may become more "intelligent" than a human being, but it doesn't mean any singularity will happen. It does not mean it can self-improve. It does not mean it can find solutions we did not think of. It's a bit hyperbolic to say "we don't know what greater intelligence can do" and conclude "it will therefore do incredible things". And to bounce on your answer, are we more intelligent than ants? Does it even make sense to say "we're more intelligent than ants"? Our differences with ants go far beyond intelligence.

harshreality 14y ago
An AI would be able to self-improve its software (unless artificially limited, and it's not clear to me whether that would be effective), and with enough initial robotic and sensory capabilities it would be able to self-improve its hardware, both computational aspects and robotic aspects. The only reasons it wouldn't are if it didn't put effort into it (too busy watching Lost), or if intelligence cannot really be improved. Humans might do the equivalent of rewriting software through thinking. But it's a more limited and slow process than could be accomplished with full read/write access to software running an AI that's completely malleable and rewritable in moments. There's plenty of evidence that intelligence can be improved. Working memory is important. We have very limited working memory. We are frequently distracted by emotions. If an AI can operate at roughly equivalent speeds with greater working memory, and if it's not distracted by emotions, it may miss some of the "finer" points of life, but I can't imagine it not leapfrogging ahead of meager human progress. Maybe it's not possible. Like the thing at the beginning of Fire Upon the Deep, if it is possible, we won't really know if it is until it exists. If it's got the temperment of a class 2 perversion or the Blight, we're toast. If it's a Power that finds us amusing or irrelevant, we might be okay.