3 ms·

I didn't address airgappedness at all, but you may still be wrong: we use the superintelligence's output, or else why would we have created it, so we are the ch

I didn't address airgappedness at all, but you may still be wrong: we use the superintelligence's output, or else why would we have created it, so we are the channel by which it affects the world. Anyway, who knows what can be done by a superintelligence which can control a stream of electrons emitted from its own CPU!

joshuamorton 10y ago
Well, exactly as much as any other cpus can do by solely controlling it's output electrons: not much. Let's not ascribe deity-like powers to things that we can understand fairly well.
- Smaug123 10y ago
  Have you ever seen a CPU running a program which is trying to communicate with the outside world through its electron side-channels? As far as I can see, your argument is "no computer has ever done this, and also we understand computers well enough to know that no computer ever will". The first clause is obvious, since no computer has ever been made to do this. The second clause is assuming without proof that we will never make a superintelligent AI. Just because you don't see how to exploit the side-channels of your system, doesn't mean they're unexploitable. This is the lesson of all security research ever.
  - joshuamorton 10y ago
    Pray explain how you could use the electrons coming out of a CPU as a side channel. I don't need anything specific, but I'd prefer something that doesn't sound like its taken out of a Heinlein novel. You're again using terms incorrectly. A "side channel" implies that someone is listening to information that is unintentionally leaked. Unless your expectation is that this CPU is going to start side channeling our minds with the EM waves its emitting (which again, "deity-like attributes"), we'd need to be specifically listening to whatever "side channel" it uses, and it would require knowledge of and access to that side channel. Something being able to send additional information over a side channel doesn't help unless that information is received, and so realistically, unless your hypothesis is "mind control/hacking the airwaves/whatever via sound waves the chip emanates" or similar, which are preposterous, it'll always be just as easy for the thing to transmit information via the normal channels.
    - Smaug123 10y ago
      A side channel is a channel through which information may leak because of the physical instantiation of an algorithm. It's not much of a stretch to include "things which let us manipulate the world" in that; do you have a better term? I thought the meaning was obvious, but apparently it's not: by "side channel" I here mean "unintended means of affecting the world by a mechanism derived from an accidental consequence of the physical implementation", by analogy with the standard "information"-related "side channel".
      - nbouscal 10y ago
        I distinctly remember a paper about AI figuring out how to either get wifi or send radio waves without access to the relevant hardware. Can't find the link at the moment though :/
        joshuamorton 10y ago
        I expect you're referring to this article: https://www.damninteresting.com/on-the-origin-of-circuits/ https://www.damninteresting.com/on-the-origin-of-circuits/
      - joshuamorton 10y ago
        I think the closest conventional thing would be a sandbox escape/backdoor. Although (not that I'm an expert) I've never heard of anything close to a sandbox escape using side-channel like things. That said, like I said, most side channel attacks are either time based, or involve things like heat and power usage of the system. The thing about all of these is that they generally allow you to get a small amount of data out that can sometimes help you with things. But again, without ascribing magic powers to the system, all the stuff that it can directly affect: power draw, temperature, disk spin speeds, monitoring LED blink speeds, noises, even the relatively insane things like EM frequency emissions can all be controlled relatively easily, and no matter how smart it is, I don't see an AGI violating physics.

joshuamorton 10y ago

Well, exactly as much as any other cpus can do by solely controlling it's output electrons: not much. Let's not ascribe deity-like powers to things that we can understand fairly well.

Smaug123 10y ago
Have you ever seen a CPU running a program which is trying to communicate with the outside world through its electron side-channels? As far as I can see, your argument is "no computer has ever done this, and also we understand computers well enough to know that no computer ever will". The first clause is obvious, since no computer has ever been made to do this. The second clause is assuming without proof that we will never make a superintelligent AI. Just because you don't see how to exploit the side-channels of your system, doesn't mean they're unexploitable. This is the lesson of all security research ever.
- joshuamorton 10y ago
  Pray explain how you could use the electrons coming out of a CPU as a side channel. I don't need anything specific, but I'd prefer something that doesn't sound like its taken out of a Heinlein novel. You're again using terms incorrectly. A "side channel" implies that someone is listening to information that is unintentionally leaked. Unless your expectation is that this CPU is going to start side channeling our minds with the EM waves its emitting (which again, "deity-like attributes"), we'd need to be specifically listening to whatever "side channel" it uses, and it would require knowledge of and access to that side channel. Something being able to send additional information over a side channel doesn't help unless that information is received, and so realistically, unless your hypothesis is "mind control/hacking the airwaves/whatever via sound waves the chip emanates" or similar, which are preposterous, it'll always be just as easy for the thing to transmit information via the normal channels.
  - Smaug123 10y ago
    A side channel is a channel through which information may leak because of the physical instantiation of an algorithm. It's not much of a stretch to include "things which let us manipulate the world" in that; do you have a better term? I thought the meaning was obvious, but apparently it's not: by "side channel" I here mean "unintended means of affecting the world by a mechanism derived from an accidental consequence of the physical implementation", by analogy with the standard "information"-related "side channel".
    - nbouscal 10y ago
      I distinctly remember a paper about AI figuring out how to either get wifi or send radio waves without access to the relevant hardware. Can't find the link at the moment though :/
      - joshuamorton 10y ago
        I expect you're referring to this article: https://www.damninteresting.com/on-the-origin-of-circuits/ https://www.damninteresting.com/on-the-origin-of-circuits/
    - joshuamorton 10y ago
      I think the closest conventional thing would be a sandbox escape/backdoor. Although (not that I'm an expert) I've never heard of anything close to a sandbox escape using side-channel like things. That said, like I said, most side channel attacks are either time based, or involve things like heat and power usage of the system. The thing about all of these is that they generally allow you to get a small amount of data out that can sometimes help you with things. But again, without ascribing magic powers to the system, all the stuff that it can directly affect: power draw, temperature, disk spin speeds, monitoring LED blink speeds, noises, even the relatively insane things like EM frequency emissions can all be controlled relatively easily, and no matter how smart it is, I don't see an AGI violating physics.