3 ms·

The claim is apparently that superhuman, savant-level intelligence could be confined to a domain of knowledge and not risk becoming generalized intelligence. I'

The claim is apparently that superhuman, savant-level intelligence could be confined to a domain of knowledge and not risk becoming generalized intelligence. I'm skeptical. If you're "only" superintelligent at language translation, or writing movies, or chess, I suspect as we ascend the tiers of increasingly super superingelligences, there's a depth of informational, structural understanding that avails itself of abstract meta principles, and meta-meta principles, and meta-meta-meta principles, and on to infinity. And that at a sufficiently high level of abstraction, something about be brilliant at translating the subtle irony of a Shakespearian sonnet into a dead tribal language is also at play in weighing strategic options in an incredibly complicated game of chess, is also at play in reading culture and finding out what kind of movie will be most successful at the box-office. I think any domain-specific intelligence, as it approximates "perfect", would independently discover and solve similar high level questions and be transferable to other domains, the way there are general principles of manufacturing that apply to multiple products. And from a sufficiently advanced perspective, "solving" chess and "solving" Shakespearan sonnet translation would look as similar to each other and panting a car red vs painting a car blue.

andreyk 7y ago
That's not the whole extent of the claim - the claim is also that these things have defined inputs and outputs as well as defined optimization procedures, and can't just go off and start doing whatever they want to maximize their utility. So even if you have a superhuman quality Google translate which can translate whole books, which may very well be able to exhibit strong intelligence, due to the API it is defined within it is still not going to go off and do nefarious things. Which strikes me as very plausible indeed.
- dmix 7y ago
  Just like factory automation it will require plenty of custom tweaking, assembling, and lining up everything in the right order with the correct timing to flow smoothly, even if the fundamental ideas are the same. Even harder is when it's not a simply multi-step process on a conveyor belt like a factory (ie, product repair vs assembly) where there are a lot of potential question marks it must adapt to.
goatlover 7y ago
The question is whether AIs will remain tools, however powerful. Or whether they will acquire their own goals and take over, for better or worse.
vermilingua 7y ago
If ImageGAN approximated perfect classification skills, would it spontaneously develop the capability to detect and evade predators? No it wouldn’t, and I think you may have missed the point. It’s about constrained domains in the mathematical sense, that there is a clearly defined scope of inputs and outputs. It would simply be impossible for an AI service to develop outside that scope.
- luc4sdreyer 7y ago
  I think glenstein is already past that point, you might be talking about different things. Drexler's solution is pretty good but it seems like a stopgap measure. Sure, if the subsystem is bound and specific there's no AGI risk, but are we just going to call it a day and stop AI research? The article seems to agree with my sentiment: > The very fact that [a Drexlerian agent] is less effective than the Bostromian agent suggests there will be pressure to build the Bostromian agent eventually (Drexler disagrees with this, but I don’t understand why)
- glenstein 7y ago
  One of the things I was really at pains to emphasize in my previous comment was that I think there are elements of information processing that are outrageously abstract, that would be familiar and accessible to AI but non-obvious to us. The transferability and generality that emerges would be nothing remotely so crude as the ImageGAN -> predator detection example that's being used to dismiss my point. An AI service that's only intended to solve one well defined problem may have pieces in its anatomy that are functionally equivalent to those in a different AI service solving a different kind of problem. You don't need to assume it's escaped the confines of it's well-defined domain, or that domains aren't well defined, for this to be true.
luc4sdreyer 7y ago
> I think any domain-specific intelligence, as it approximates "perfect", would independently discover and solve similar high level questions and be transferable to other domains There is a trap here, in the sense that we tend to lump all the unsolved AI problems into to the "requires AGI" bucket. Some computer scientists thought that a program that could beat a human in chess would have to be an AGI (can't find a source for this, can someone help?). Language translation that's good enough to convey basic meaning is another one that turned out to be _much_ easier to do than what was suspected. I'm still dumbfounded by GPT-2 and suddenly the Turing test doesn't seem that far out of reach as it used to. What if 99% of human intellectual tasks can be solved by domain-specific intelligence? What if we're not as smart as we think we are? That seems to be a trend. On the other hand, there are some tasks that cannot be subdivided, and necessarily requires a massive amount of context about human culture, the real world, and excellent logical reason ability. The program that links the domain-specific programs might have to be like that, and then we're back at something that looks _very much_ like an AGI.
solipsism 7y ago
would independently discover and solve similar high level questions and be transferable to other domains From where would this curiosity and desire to find analogies and to use knowledge across domains arise? Are you saying it would spontaneously appear? You're going to have to explain how I think. How would a chess machine suddenly aquire a taste for sonnets? These traits have obvious advantages that led to their being selected for in humans. In an AI we'd presumably have to program it in purposefully.
- glenstein 7y ago
  >From where would this curiosity and desire to find analogies and to use knowledge across domains arise? Are you saying it would spontaneously appear? From the perspective of an AI it wouldn't at all be some new desire or curiosity for something outside its defined domain. It would be something squarely within its own domain that nevertheless has applicability outside of it.
  - solipsism 7y ago
    It would be something squarely within its own domain that nevertheless has applicability outside of it Why would it step outside its own domain to make this realization?
    - glenstein 7y ago
      You're continuing to attribute claims to me that I don't believe I'm making. If you think a freestanding statement about "applicability" in a general sense needs to be construed as an AI actively stepping outside it's domain, and as AI having it's own "realization", you should support that contention, so I can understand why you think that that's what I'm saying and we can go from there.
      - solipsism 7y ago
        We're talking about the risk of an AI going rogue and doing/thinking things we didn't want it to. That's why we're here. If all you're saying is, "an AI may learn things that might be applicable to other domains", that's fine but it doesn't support your point that you don't think AI will be easily confined to a single domain of knowledge. It's entirely possible a chess AI would be potentially incredibly good at some other domain like murdering humans (say), but you're going to have to explain why it would acquire a taste for murdering humans if you're saying we should be worried. I think any domain-specific intelligence, as it approximates "perfect", would independently discover and solve similar high level questions Solving a question requires intent. What would prompt it to consider those questions? If you're not saying those questions are in other domains, how is it relevant?

andreyk 7y ago

That's not the whole extent of the claim - the claim is also that these things have defined inputs and outputs as well as defined optimization procedures, and can't just go off and start doing whatever they want to maximize their utility. So even if you have a superhuman quality Google translate which can translate whole books, which may very well be able to exhibit strong intelligence, due to the API it is defined within it is still not going to go off and do nefarious things. Which strikes me as very plausible indeed.

dmix 7y ago
Just like factory automation it will require plenty of custom tweaking, assembling, and lining up everything in the right order with the correct timing to flow smoothly, even if the fundamental ideas are the same. Even harder is when it's not a simply multi-step process on a conveyor belt like a factory (ie, product repair vs assembly) where there are a lot of potential question marks it must adapt to.

goatlover 7y ago

The question is whether AIs will remain tools, however powerful. Or whether they will acquire their own goals and take over, for better or worse.

vermilingua 7y ago

If ImageGAN approximated perfect classification skills, would it spontaneously develop the capability to detect and evade predators? No it wouldn’t, and I think you may have missed the point. It’s about constrained domains in the mathematical sense, that there is a clearly defined scope of inputs and outputs. It would simply be impossible for an AI service to develop outside that scope.

luc4sdreyer 7y ago
I think glenstein is already past that point, you might be talking about different things. Drexler's solution is pretty good but it seems like a stopgap measure. Sure, if the subsystem is bound and specific there's no AGI risk, but are we just going to call it a day and stop AI research? The article seems to agree with my sentiment: > The very fact that [a Drexlerian agent] is less effective than the Bostromian agent suggests there will be pressure to build the Bostromian agent eventually (Drexler disagrees with this, but I don’t understand why)
glenstein 7y ago
One of the things I was really at pains to emphasize in my previous comment was that I think there are elements of information processing that are outrageously abstract, that would be familiar and accessible to AI but non-obvious to us. The transferability and generality that emerges would be nothing remotely so crude as the ImageGAN -> predator detection example that's being used to dismiss my point. An AI service that's only intended to solve one well defined problem may have pieces in its anatomy that are functionally equivalent to those in a different AI service solving a different kind of problem. You don't need to assume it's escaped the confines of it's well-defined domain, or that domains aren't well defined, for this to be true.

luc4sdreyer 7y ago

> I think any domain-specific intelligence, as it approximates "perfect", would independently discover and solve similar high level questions and be transferable to other domains There is a trap here, in the sense that we tend to lump all the unsolved AI problems into to the "requires AGI" bucket. Some computer scientists thought that a program that could beat a human in chess would have to be an AGI (can't find a source for this, can someone help?). Language translation that's good enough to convey basic meaning is another one that turned out to be _much_ easier to do than what was suspected. I'm still dumbfounded by GPT-2 and suddenly the Turing test doesn't seem that far out of reach as it used to. What if 99% of human intellectual tasks can be solved by domain-specific intelligence? What if we're not as smart as we think we are? That seems to be a trend. On the other hand, there are some tasks that cannot be subdivided, and necessarily requires a massive amount of context about human culture, the real world, and excellent logical reason ability. The program that links the domain-specific programs might have to be like that, and then we're back at something that looks _very much_ like an AGI.

solipsism 7y ago

would independently discover and solve similar high level questions and be transferable to other domains From where would this curiosity and desire to find analogies and to use knowledge across domains arise? Are you saying it would spontaneously appear? You're going to have to explain how I think. How would a chess machine suddenly aquire a taste for sonnets? These traits have obvious advantages that led to their being selected for in humans. In an AI we'd presumably have to program it in purposefully.

glenstein 7y ago
>From where would this curiosity and desire to find analogies and to use knowledge across domains arise? Are you saying it would spontaneously appear? From the perspective of an AI it wouldn't at all be some new desire or curiosity for something outside its defined domain. It would be something squarely within its own domain that nevertheless has applicability outside of it.
- solipsism 7y ago
  It would be something squarely within its own domain that nevertheless has applicability outside of it Why would it step outside its own domain to make this realization?
  - glenstein 7y ago
    You're continuing to attribute claims to me that I don't believe I'm making. If you think a freestanding statement about "applicability" in a general sense needs to be construed as an AI actively stepping outside it's domain, and as AI having it's own "realization", you should support that contention, so I can understand why you think that that's what I'm saying and we can go from there.
    - solipsism 7y ago
      We're talking about the risk of an AI going rogue and doing/thinking things we didn't want it to. That's why we're here. If all you're saying is, "an AI may learn things that might be applicable to other domains", that's fine but it doesn't support your point that you don't think AI will be easily confined to a single domain of knowledge. It's entirely possible a chess AI would be potentially incredibly good at some other domain like murdering humans (say), but you're going to have to explain why it would acquire a taste for murdering humans if you're saying we should be worried. I think any domain-specific intelligence, as it approximates "perfect", would independently discover and solve similar high level questions Solving a question requires intent. What would prompt it to consider those questions? If you're not saying those questions are in other domains, how is it relevant?