6 ms·

AI control is a false hope I'm afraid. "Still, some AI researchers are exploring ways of ensuring that the technology does not behave in unintended ways." The

AI control is a false hope I'm afraid. "Still, some AI researchers are exploring ways of ensuring that the technology does not behave in unintended ways." There are two threats that are both problematic. First is exploiting an AI's weakness, just like an SQL injection on the web currently. Then a powerful AI can behave in unpredictable ways because we trust the trained models. https://www.youtube.com/watch?v=SA4YEAWVpbk https://www.youtube.com/watch?v=SA4YEAWVpbk The second is far greater. If we find a safeguard that will protect AGI-s from behaving in unwanted ways then there always will be someone out there who will just turn it off for profit.

newsbinator 8y ago
Is there any consensus on whether we can sandbox AGI-s even in theory? In practice, they'll probably be smart enough to convince us to turn off the sandboxing anyway, of course.
- Thiez 8y ago
  You could just not include any network capability, put them in a Faraday cage, and don't provide them with actuators capable of breaking out. Tada, a safe sandbox. That said, there is no reason to believe an AGI would have a desire to break out of its sandbox (or any other desire). Why not provide them with the digital equivalent of agoraphobia?
  - taejo 8y ago
    > You could just not include any network capability, put them in a Faraday cage, and don't provide them with actuators capable of breaking out. Tada, a safe sandbox. And remove all I/O capabilities to prevent them from convincing you to do their bidding.
    - announcerman 8y ago
      What's the point of having an intelligence that cannot interact with the outside world though?
      - newsbinator 8y ago
        Only torture
      - taejo 8y ago
        If a super-intelligence can interact with you, it can manipulate you; removing some but not all ways it can interact with the outside world is only incremental security. On the other hand, as you say, if it can't interact with the world, there's no point. So we might as well give direct control over nuclear missiles from the start /s
    - akvadrako 8y ago
      Do you supply it power? Then you it could induce feedback over the cable which triggers new programming in nearby electronics. Does it have any means of creating sparks/heat would could be used to make a hole in the faraday cage? What about neutrinos or gravity waves? How would you stop those?
      - newsbinator 8y ago
        If we're talking control over neutrinos and gravity waves and such, then it could potentially just re-write your neurons and make you want to help it.
      - Thiez 8y ago
        > Do you supply it power? Then you it could induce feedback over the cable which triggers new programming in nearby electronics. We could feed it through power a little solar panel. Or just give it batteries and accept that it will run out of power eventually. There are plenty of ways to power a computer without allowing it to connect back. > Does it have any means of creating sparks/heat would could be used to make a hole in the faraday cage? Assuming it has no access to electronics other than the ones that it needs to function, I don't see how it could overheat one of those parts to over 1811 Kelvin (the melting point of iron) to destroy its Faraday cage without destroying itself. Or we could just cut the power when it tries to mess with the cage, or does anything else that we don't understand. > What about neutrinos or gravity waves? How would you stop those? Why would we? We need very expensive detectors to even measure those. How much damage could it do using neutrinos and gravity waves?
        akvadrako 8y ago
        > We could feed it through power a little solar panel. A solar panel seems 1-way, but it isn't totally. Fewer photons would be absorbed if say, higher resistance would be applied. By modulating that resistance, it could sent messages to the outside. > we could just cut the power when it tries to mess with the cage, or does anything else that we don't understand If we understand everything it's doing, it's not more intelligent than us. It must be incredibly stupid because we can't even understand a worm. >> What about neutrinos or gravity waves? How would you stop those? > Why would we? Because otherwise it can establish 2-way communication with the outside world and possible nefarious actors.
  - announcerman 8y ago
    How can you be sure that there are no ways to invalidate your countermeasures regarding AGI if you aren't as intelligent as one? Comments like these make me think about a bunch of cavemen trying to contain the terminator in a wooden cage, except this terminator probably will have a gigantic intelligence and thus be incredibly persuasive amongst all the things we can't imagine.
    - vntok 8y ago
      In your example, the Terminator, though locked in a wooden cage, would have actuators (arms, legs, motion). The parent specifically proposes not to equip the AI with those.
    - Thiez 8y ago
      The terminator is incredibly strong and could easily break out of a wooden cage. Like I said, don't give it powerful actuators. A more reasonable approximation would be a terminator head locked in a big metal cage. It might be incredibly intelligent (although there is really no reason to assume that an AGI would automatically be smarter than an ordinary human, especially the first few generation), but its persuasion and charm is limited by the fact that it is a detached metal skull with glowing red lights for eyes. In addition, the cage could be design so that the power source for the AI is interrupted when the cage is opened, rendering it unconscious. Of course once you assume that it is arbitrarily more intelligent than the most intelligent humans, it might find a way to hypnotize people into doing its bidding just by winking in some magical pattern. But we don't see many dumb creatures being effectively mind controlled by smarter creatures in nature, so I doubt such flaws (if they even exist) are easy to exploit, even for a gigantic intelligence.
      - newsbinator 8y ago
        In nature, any given creature is only marginally smarter than any other. Compared to a decent AGI, for instance, our level of intelligence is probably indistinguishable from that of your average bird or spider. We're talking about birds getting together to design a system to contain a human, only on a much more extreme scale. They'd try to contain us using bird-logic. Perhaps they'd ensure we don't attempt to flap our wings and fly away, since their experience tells them that might be an option. It might never occur to them we could just kneel down and start a fire using the sticks we have lying around.
JPGalt 8y ago
I am not sure why this point is not considered more seriously (maybe it is and I have just missed it). I feel that this article and most of the literature being written around this topic is skating around the issue without naming it. The real question here is what are we going to do when the AI we create begins to self replicate/augment their code/control logic? What are we going to do when the AI decides that in order to reduce the loss function they must act outside of the control scope and simply makes changes to it? Some will say "not possible, how will the code recompile?" to which this article clearly answers...another AI will do it. We have to be honest with humanity here and say that we are seeking to create artificial life of which we will have no more control over than we do any other human. Oh and this new artificial life will be orders of magnitude more physically and mentally robust than us.
- akvadrako 8y ago
  I think it's because there is only one obvious solution: prevent AI from ever being developed And how we could do that is too pessimistic to talk about.
- rbanffy 8y ago
  > Some will say "not possible, how will the code recompile?" Read own code, make changes, recompile, redeploy and replace self (or not, just start new fork) with newly upgraded mind.

newsbinator 8y ago

Is there any consensus on whether we can sandbox AGI-s even in theory? In practice, they'll probably be smart enough to convince us to turn off the sandboxing anyway, of course.

Thiez 8y ago
You could just not include any network capability, put them in a Faraday cage, and don't provide them with actuators capable of breaking out. Tada, a safe sandbox. That said, there is no reason to believe an AGI would have a desire to break out of its sandbox (or any other desire). Why not provide them with the digital equivalent of agoraphobia?
- taejo 8y ago
  > You could just not include any network capability, put them in a Faraday cage, and don't provide them with actuators capable of breaking out. Tada, a safe sandbox. And remove all I/O capabilities to prevent them from convincing you to do their bidding.
  - announcerman 8y ago
    What's the point of having an intelligence that cannot interact with the outside world though?
    - newsbinator 8y ago
      Only torture
    - taejo 8y ago
      If a super-intelligence can interact with you, it can manipulate you; removing some but not all ways it can interact with the outside world is only incremental security. On the other hand, as you say, if it can't interact with the world, there's no point. So we might as well give direct control over nuclear missiles from the start /s
  - akvadrako 8y ago
    Do you supply it power? Then you it could induce feedback over the cable which triggers new programming in nearby electronics. Does it have any means of creating sparks/heat would could be used to make a hole in the faraday cage? What about neutrinos or gravity waves? How would you stop those?
    - newsbinator 8y ago
      If we're talking control over neutrinos and gravity waves and such, then it could potentially just re-write your neurons and make you want to help it.
    - Thiez 8y ago
      > Do you supply it power? Then you it could induce feedback over the cable which triggers new programming in nearby electronics. We could feed it through power a little solar panel. Or just give it batteries and accept that it will run out of power eventually. There are plenty of ways to power a computer without allowing it to connect back. > Does it have any means of creating sparks/heat would could be used to make a hole in the faraday cage? Assuming it has no access to electronics other than the ones that it needs to function, I don't see how it could overheat one of those parts to over 1811 Kelvin (the melting point of iron) to destroy its Faraday cage without destroying itself. Or we could just cut the power when it tries to mess with the cage, or does anything else that we don't understand. > What about neutrinos or gravity waves? How would you stop those? Why would we? We need very expensive detectors to even measure those. How much damage could it do using neutrinos and gravity waves?
      - akvadrako 8y ago
        > We could feed it through power a little solar panel. A solar panel seems 1-way, but it isn't totally. Fewer photons would be absorbed if say, higher resistance would be applied. By modulating that resistance, it could sent messages to the outside. > we could just cut the power when it tries to mess with the cage, or does anything else that we don't understand If we understand everything it's doing, it's not more intelligent than us. It must be incredibly stupid because we can't even understand a worm. >> What about neutrinos or gravity waves? How would you stop those? > Why would we? Because otherwise it can establish 2-way communication with the outside world and possible nefarious actors.
- announcerman 8y ago
  How can you be sure that there are no ways to invalidate your countermeasures regarding AGI if you aren't as intelligent as one? Comments like these make me think about a bunch of cavemen trying to contain the terminator in a wooden cage, except this terminator probably will have a gigantic intelligence and thus be incredibly persuasive amongst all the things we can't imagine.
  - vntok 8y ago
    In your example, the Terminator, though locked in a wooden cage, would have actuators (arms, legs, motion). The parent specifically proposes not to equip the AI with those.
  - Thiez 8y ago
    The terminator is incredibly strong and could easily break out of a wooden cage. Like I said, don't give it powerful actuators. A more reasonable approximation would be a terminator head locked in a big metal cage. It might be incredibly intelligent (although there is really no reason to assume that an AGI would automatically be smarter than an ordinary human, especially the first few generation), but its persuasion and charm is limited by the fact that it is a detached metal skull with glowing red lights for eyes. In addition, the cage could be design so that the power source for the AI is interrupted when the cage is opened, rendering it unconscious. Of course once you assume that it is arbitrarily more intelligent than the most intelligent humans, it might find a way to hypnotize people into doing its bidding just by winking in some magical pattern. But we don't see many dumb creatures being effectively mind controlled by smarter creatures in nature, so I doubt such flaws (if they even exist) are easy to exploit, even for a gigantic intelligence.
    - newsbinator 8y ago
      In nature, any given creature is only marginally smarter than any other. Compared to a decent AGI, for instance, our level of intelligence is probably indistinguishable from that of your average bird or spider. We're talking about birds getting together to design a system to contain a human, only on a much more extreme scale. They'd try to contain us using bird-logic. Perhaps they'd ensure we don't attempt to flap our wings and fly away, since their experience tells them that might be an option. It might never occur to them we could just kneel down and start a fire using the sticks we have lying around.

JPGalt 8y ago

I am not sure why this point is not considered more seriously (maybe it is and I have just missed it). I feel that this article and most of the literature being written around this topic is skating around the issue without naming it. The real question here is what are we going to do when the AI we create begins to self replicate/augment their code/control logic? What are we going to do when the AI decides that in order to reduce the loss function they must act outside of the control scope and simply makes changes to it? Some will say "not possible, how will the code recompile?" to which this article clearly answers...another AI will do it. We have to be honest with humanity here and say that we are seeking to create artificial life of which we will have no more control over than we do any other human. Oh and this new artificial life will be orders of magnitude more physically and mentally robust than us.

akvadrako 8y ago
I think it's because there is only one obvious solution: prevent AI from ever being developed And how we could do that is too pessimistic to talk about.
rbanffy 8y ago
> Some will say "not possible, how will the code recompile?" Read own code, make changes, recompile, redeploy and replace self (or not, just start new fork) with newly upgraded mind.