4 ms·

> The fears of machine superintelligence are based on the belief that true AI is just around a corner. No, the fear of AGI is that the time between when we fir

> The fears of machine superintelligence are based on the belief that true AI is just around a corner. No, the fear of AGI is that the time between when we first start to see promising progess in AI to the point where AI is out of our control would be so short that we would not be able to react to prevent it.

maaaats 12y ago
But the fear and thoughts should instead be directed to how one can solve the economical an social challenges that arise when AI removes a lot of jobs. That is the issue here, not some Skynet-thingy destroying humanity.
- foobarqux 12y ago
  They are both legitimate problems.
eli_gottlieb 12y ago
>the time between when we first start to see promising progess in AI to the point where AI is out of our control would be so short that we would not be able to react to prevent it. That seems pretty absurd to me. A human-level AGI is going to require a sizable amount of time and data to train, so that it knows something rather than nothing. In fact, getting to the point where self-improvement becomes viable will take a good deal longer than training/educational periods for most other tasks. The information complexity is just higher. So while AI safety should be taken seriously, we do need to "sober up" by remembering that an AGI has to obey the rules of information theory.
- foobarqux 12y ago
  What's absurd is to have a strong conviction about inviolable restrictions on systems that haven't been imagined yet. The examples you cite aren't even compelling on their face. For instance if you include whole-brain emulations in AGI (and we should because the consequences are essentially the same) it would take zero time to train. But even without whole-brain emulations a child learns rapidly from its environment alone. An AGI could conceivably do the same but faster, and moreover could take advantage of access to the Internet. I don't know what information theory has to do with any of that.
  - eli_gottlieb 12y ago
    >What's absurd is to have a strong conviction about inviolable restrictions on systems that haven't been imagined yet. AGI has already been imagined. And yes, it does have to obey the laws of information theory: only one bit of knowledge can be learned from sensory inputs with one bit of entropy. >An AGI could conceivably do the same but faster, and moreover could take advantage of access to the Internet. Children still take 20 years to grow from infancy to being useful adults. If we hypothesize that an AGI could learn, say, 10x faster, we're assuming a two-year training time.
    - foobarqux 12y ago
      All the manifestations of AGI have not been imagined, and even ones that have like WBE refute your claims. Even if you could somehow create a model precise enough to talk about information theory (define a symbol universe, map information to knowledge, itself a not-well defined term) it would not seem to be restrictive. Again, children take in a massive amount of sensory input. Finally, the singularity hypothesis is that AGI could learn exponentially faster, not merely 10x. And again, WBEs don't need to learn at all.

maaaats 12y ago

But the fear and thoughts should instead be directed to how one can solve the economical an social challenges that arise when AI removes a lot of jobs. That is the issue here, not some Skynet-thingy destroying humanity.

foobarqux 12y ago
They are both legitimate problems.

eli_gottlieb 12y ago

>the time between when we first start to see promising progess in AI to the point where AI is out of our control would be so short that we would not be able to react to prevent it. That seems pretty absurd to me. A human-level AGI is going to require a sizable amount of time and data to train, so that it knows something rather than nothing. In fact, getting to the point where self-improvement becomes viable will take a good deal longer than training/educational periods for most other tasks. The information complexity is just higher. So while AI safety should be taken seriously, we do need to "sober up" by remembering that an AGI has to obey the rules of information theory.

foobarqux 12y ago
What's absurd is to have a strong conviction about inviolable restrictions on systems that haven't been imagined yet. The examples you cite aren't even compelling on their face. For instance if you include whole-brain emulations in AGI (and we should because the consequences are essentially the same) it would take zero time to train. But even without whole-brain emulations a child learns rapidly from its environment alone. An AGI could conceivably do the same but faster, and moreover could take advantage of access to the Internet. I don't know what information theory has to do with any of that.
- eli_gottlieb 12y ago
  >What's absurd is to have a strong conviction about inviolable restrictions on systems that haven't been imagined yet. AGI has already been imagined. And yes, it does have to obey the laws of information theory: only one bit of knowledge can be learned from sensory inputs with one bit of entropy. >An AGI could conceivably do the same but faster, and moreover could take advantage of access to the Internet. Children still take 20 years to grow from infancy to being useful adults. If we hypothesize that an AGI could learn, say, 10x faster, we're assuming a two-year training time.
  - foobarqux 12y ago
    All the manifestations of AGI have not been imagined, and even ones that have like WBE refute your claims. Even if you could somehow create a model precise enough to talk about information theory (define a symbol universe, map information to knowledge, itself a not-well defined term) it would not seem to be restrictive. Again, children take in a massive amount of sensory input. Finally, the singularity hypothesis is that AGI could learn exponentially faster, not merely 10x. And again, WBEs don't need to learn at all.