6 ms·

As far as I can tell, the claims of the Singularity are as follows: Human-level AI is possible. Human-level AI will likely be practical this century. Once AI

As far as I can tell, the claims of the Singularity are as follows: Human-level AI is possible. Human-level AI will likely be practical this century. Once AI is sufficiently advanced, it will be capable of AI research. Most terminal goals would lead to intelligence enhancement as a sub goal, as any system with a goal would be better able to pursue it if more intelligent. Therefore, most AIs will devote resources to increasing their intelligence. Assuming humans are not near the maximum possible intellectual level, super-human AI will likely be developed soon after an AI is created that is as smart as the smartest human. This means the future will not be controlled by humans, as we will no longer be the smartest things in town. Unless we can predict the goals of the AI, we should consider the future unpredictable after the point of its creation, as our models of the future rely on human-shaped agents with human goals and capabilities. With these assumptions broken, we have a sort of singularity, a breakdown of our models of the future. It's quite conjunctive, but this all seems pretty reasonable to me - and it's worth spending resources studying.

eli_gottlieb 12y ago
That assumes AIs can enhance their intelligence given fixed computational resources. We don't actually know the shape of the curve for optimization power on the X-axis and necessary FLOPs on the Y-axis. If it's anything larger than linear, AIs will not self-improve in a "singularity" type fashion, but in fact hit a point of severely diminished returns relatively quickly. Judging by the computational complexity of machine-learning algorithms, it's actually entirely possible that curve is exponential (meaning that AGI is DEXPTIME-complete), in which case the only question is how far we are towards the diminishing-returns point.
- viraptor 12y ago
  But that assumes running AI on some architecture where FLOPs have any real meaning. If we grow an AI that's externally accessible, is that less of an achievement than simulating the same on a general purpose computer? Still assuming it could achieve singularity that is...
- ThrustVectoring 12y ago
  This objection further assumes that the end-point after consuming all low-hanging fruit is a pre-singularity level AI. If a self-improving AI runs into algorithmic limits only after figuring out how to turn the solar system into computronium, that's still worth calling a "singularity".
- baddox 12y ago
  Does the singularity concept actually require some sort of quickly growing or even asymptotic curve for intelligence improvement? I don't see why that would be a requirement. Even if AIs are only slightly more intelligent than humans, it seems like many of the effects associated with the singularity could occur. If AIs were more intelligent than humans to the same extent humans are more intelligent than the next most intelligent species on Earth, surely we would consider that to resemble the proposed singularity.
- webmaven 12y ago
  Except that we already have examples of semi-autonomous non-AGI software that successfully acquires additional computational resources via parasitism. Further, beyond the task of software improvement, the task of improving hardware is itself already heavily software-mediated when performed by humans, and will likely be a prime candidate for applying pre-AGI AI. There are likely other ways to attack the problem, such as not focusing on hardware improvements per-se, but instead on the cycle time for getting the improvements deployed (eg. somehow eliminating or cutting the size/cost/construction time of new Fabs).
worldsayshi 12y ago
This idea of singularity relies on the assumption that intelligence scales easily. If we have a machine that is more intelligent than the average human, or even more intelligent than the most intelligent human. Will it be more intelligent than a group of intelligent people? How many? If you have an AI running on a supercomputer that is more intelligent than the most intelligent human you might still need a few billion supercomputers to outmatch the human race.
- eli_gottlieb 12y ago
  This assumes that a single greater-than-human AI doesn't figure out how to get humans onto its side. Given that it will not be a cartoon supervillain, that task will actually be incredibly easy: find ways to give humans what they want in exchange for doing what the AI wants.
  - worldsayshi 12y ago
    Sure, then it will become just another productive person/citizen. We work together because we benefit from it. Working against another intelligent being is significantly harder in the long term. edit: And the collective is significantly more intelligent than the individual (for some but definitely not all reasonable definitions of intelligent and collective).
    - eli_gottlieb 12y ago
      >Sure, then it will become just another productive person/citizen. Only in the sense that Stalin or Pinochet was a productive citizen. Powerful people can mobilize others to their own ends without those ends being at all good or moral for everyone who is not so mobilized.
      - BEEdwards 12y ago
        You can disagree with his methods, but saying Stalin wasn't productive is just plain wrong. From agrarians to victors of ww2 in ~20 years is pretty impressive by any measure.
malka 12y ago
> Human-level AI is possible. > Human-level AI will likely be practical this century. Both are unknown.
- JamesArgo 12y ago
  >Human-level AI is possible. Well we exist, so we know human-level intelligence can be implemented in physics. So we certainly know AI is possible.
  - malka 12y ago
    Playing a little bit devil advocate here ;) We have no mean to know. Maybe the subjective experience is an inherent part of our intelligence, and not just a side effect. And I do not see any way of applying the scientific method to the subjective experience. Nethertheless, I really hope that the future will prove me wrong.
    - sampo 12y ago
      Please correct if I'm wrong, but sounds like you think human intelligence requires something outside of known physics, something "magical", to work?
      - malka 12y ago
        I'm agnostic about this question. We do not have the necessarry tools to answer right now. One possible way I can see to do that would be to map a human brain neuron by neuron and simulate it. We are not here yet.
  - sampo 12y ago
    I agree to that. But the question of whether human-level I is capable of designing human-level AI remains open still.
    - dragonwriter 12y ago
sampo 12y ago
> Human-level AI will likely be practical this century. Given how far away from that goal we are now, after 85 previous years of research, I would not hold my breath for reaching that goal in the next 85 years.
- saalweachter 12y ago
  While I'm in the camp of "people who say 'X will happen in the next Y years' are really saying 'X will happen before I die'", I think it is unreasonable to treat the last 50-85 years as a failure for AI. There are two things holding back AI (which from a singularity-standpoint just means "a machine capable of general problem-solving", and not "a machine possessing the ineffable characteristics of human consciousness"). First, we don't know how. Second, we might not have the computational capacity. While we still don't know how, the bar is getting lower and lower as time goes on. Increasing computational capacity means that we can start with worse and worse initial versions of a general intelligence. Maybe a really special genius could write a general problem solver which ran on a 386 (even if it lacked the self awareness to feel the pain of running on a 386). Maybe we're too dumb to ever write a general problem solver even if we had an infinately fast computer with unlimited memory. But even if programmers have made no progress since the 1960s, we are definitely not in the same position.
  - mcguire 12y ago
    "While I'm in the camp of "people who say 'X will happen in the next Y years' are really saying 'X will happen before I die'"..." That depends on who is saying it. I've noticed that when actual researchers say, "We hope this will provide a treatment for reticularly homicidal brain-worms within 5 years", they mean they have no idea if or when any such practical material benefits will appear. If they say, "We will be able to get effectively unlimited amounts of energy from russet potatos within 20 years," they mean they don't believe it's ever going to happen. If the difficulty is exponential, as I rather expect it to be (in my own completely unfounded opinion), then lowering the bar a given amount doesn't help much. And as far as increasing computational capacity goes, we're already deep into the realm where communication costs outweigh actual computational costs, which makes it a whole different ball game going forward. If you want a further speculation, I suspect that modern statistical methods like those behind th' Goog, self-driving cars, and what-not will not get too much closer to a super intelligence than conventional heuristic search. They're capable of doing a lot of things easily, that only humans could do heretofore, but they're not actually more powerful. So, "a machine capable of general problem-solving" is easily conceivable in the same sense that you are capable of general problem solving, but I can't see a singularity on the other side of that.

eli_gottlieb 12y ago

That assumes AIs can enhance their intelligence given fixed computational resources. We don't actually know the shape of the curve for optimization power on the X-axis and necessary FLOPs on the Y-axis. If it's anything larger than linear, AIs will not self-improve in a "singularity" type fashion, but in fact hit a point of severely diminished returns relatively quickly. Judging by the computational complexity of machine-learning algorithms, it's actually entirely possible that curve is exponential (meaning that AGI is DEXPTIME-complete), in which case the only question is how far we are towards the diminishing-returns point.

viraptor 12y ago
But that assumes running AI on some architecture where FLOPs have any real meaning. If we grow an AI that's externally accessible, is that less of an achievement than simulating the same on a general purpose computer? Still assuming it could achieve singularity that is...
ThrustVectoring 12y ago
This objection further assumes that the end-point after consuming all low-hanging fruit is a pre-singularity level AI. If a self-improving AI runs into algorithmic limits only after figuring out how to turn the solar system into computronium, that's still worth calling a "singularity".
baddox 12y ago
Does the singularity concept actually require some sort of quickly growing or even asymptotic curve for intelligence improvement? I don't see why that would be a requirement. Even if AIs are only slightly more intelligent than humans, it seems like many of the effects associated with the singularity could occur. If AIs were more intelligent than humans to the same extent humans are more intelligent than the next most intelligent species on Earth, surely we would consider that to resemble the proposed singularity.
webmaven 12y ago
Except that we already have examples of semi-autonomous non-AGI software that successfully acquires additional computational resources via parasitism. Further, beyond the task of software improvement, the task of improving hardware is itself already heavily software-mediated when performed by humans, and will likely be a prime candidate for applying pre-AGI AI. There are likely other ways to attack the problem, such as not focusing on hardware improvements per-se, but instead on the cycle time for getting the improvements deployed (eg. somehow eliminating or cutting the size/cost/construction time of new Fabs).

worldsayshi 12y ago

This idea of singularity relies on the assumption that intelligence scales easily. If we have a machine that is more intelligent than the average human, or even more intelligent than the most intelligent human. Will it be more intelligent than a group of intelligent people? How many? If you have an AI running on a supercomputer that is more intelligent than the most intelligent human you might still need a few billion supercomputers to outmatch the human race.

eli_gottlieb 12y ago
This assumes that a single greater-than-human AI doesn't figure out how to get humans onto its side. Given that it will not be a cartoon supervillain, that task will actually be incredibly easy: find ways to give humans what they want in exchange for doing what the AI wants.
- worldsayshi 12y ago
  Sure, then it will become just another productive person/citizen. We work together because we benefit from it. Working against another intelligent being is significantly harder in the long term. edit: And the collective is significantly more intelligent than the individual (for some but definitely not all reasonable definitions of intelligent and collective).
  - eli_gottlieb 12y ago
    >Sure, then it will become just another productive person/citizen. Only in the sense that Stalin or Pinochet was a productive citizen. Powerful people can mobilize others to their own ends without those ends being at all good or moral for everyone who is not so mobilized.
    - BEEdwards 12y ago
      You can disagree with his methods, but saying Stalin wasn't productive is just plain wrong. From agrarians to victors of ww2 in ~20 years is pretty impressive by any measure.

malka 12y ago

> Human-level AI is possible. > Human-level AI will likely be practical this century. Both are unknown.

JamesArgo 12y ago
>Human-level AI is possible. Well we exist, so we know human-level intelligence can be implemented in physics. So we certainly know AI is possible.
- malka 12y ago
  Playing a little bit devil advocate here ;) We have no mean to know. Maybe the subjective experience is an inherent part of our intelligence, and not just a side effect. And I do not see any way of applying the scientific method to the subjective experience. Nethertheless, I really hope that the future will prove me wrong.
  - sampo 12y ago
    Please correct if I'm wrong, but sounds like you think human intelligence requires something outside of known physics, something "magical", to work?
    - malka 12y ago
      I'm agnostic about this question. We do not have the necessarry tools to answer right now. One possible way I can see to do that would be to map a human brain neuron by neuron and simulate it. We are not here yet.
- sampo 12y ago
  I agree to that. But the question of whether human-level I is capable of designing human-level AI remains open still.
  - dragonwriter 12y ago

sampo 12y ago

> Human-level AI will likely be practical this century. Given how far away from that goal we are now, after 85 previous years of research, I would not hold my breath for reaching that goal in the next 85 years.

saalweachter 12y ago
While I'm in the camp of "people who say 'X will happen in the next Y years' are really saying 'X will happen before I die'", I think it is unreasonable to treat the last 50-85 years as a failure for AI. There are two things holding back AI (which from a singularity-standpoint just means "a machine capable of general problem-solving", and not "a machine possessing the ineffable characteristics of human consciousness"). First, we don't know how. Second, we might not have the computational capacity. While we still don't know how, the bar is getting lower and lower as time goes on. Increasing computational capacity means that we can start with worse and worse initial versions of a general intelligence. Maybe a really special genius could write a general problem solver which ran on a 386 (even if it lacked the self awareness to feel the pain of running on a 386). Maybe we're too dumb to ever write a general problem solver even if we had an infinately fast computer with unlimited memory. But even if programmers have made no progress since the 1960s, we are definitely not in the same position.
- mcguire 12y ago
  "While I'm in the camp of "people who say 'X will happen in the next Y years' are really saying 'X will happen before I die'"..." That depends on who is saying it. I've noticed that when actual researchers say, "We hope this will provide a treatment for reticularly homicidal brain-worms within 5 years", they mean they have no idea if or when any such practical material benefits will appear. If they say, "We will be able to get effectively unlimited amounts of energy from russet potatos within 20 years," they mean they don't believe it's ever going to happen. If the difficulty is exponential, as I rather expect it to be (in my own completely unfounded opinion), then lowering the bar a given amount doesn't help much. And as far as increasing computational capacity goes, we're already deep into the realm where communication costs outweigh actual computational costs, which makes it a whole different ball game going forward. If you want a further speculation, I suspect that modern statistical methods like those behind th' Goog, self-driving cars, and what-not will not get too much closer to a super intelligence than conventional heuristic search. They're capable of doing a lot of things easily, that only humans could do heretofore, but they're not actually more powerful. So, "a machine capable of general problem-solving" is easily conceivable in the same sense that you are capable of general problem solving, but I can't see a singularity on the other side of that.